[PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

The FPGA Image Load framework provides an API to upload image
files to an FPGA device. Image files are self-describing. They could
contain FPGA images, BMC images, Root Entry Hashes, or other device
specific files. It is up to the lower-level device driver and the
target device to authenticate and disposition the file data.

The n3000bmc-sec-update driver is the first driver to use the FPGA Image
Load Framework. This driver was previously submitted in the same patch
set, but was split into a separate patch set starting with v2.

Changelog v16 -> v17:
 - Documentation cleanup with clarification for changes to how cancellation
   is handled.
 - Removed the FPGA_IMAGE_ERR_NONE macro in favor of returning zero. The
   constant name made sense when we were using enum data types, but is no
   longer necessary.
 - Changed the syntax of the write_blk() function and renamed it to
   write().
   It now returns a positive size or a negative error number.
 - Removed the default block size for write_block(). The lower-level driver
   must choose a block size.
 - Changed the prepare() and write() ops to pass in elements of the
   fpga_image_load data structure instead of having the op functions access
   them directly via imgld.
 - Changed work queue from system_unbound_wq to system_long_wq.
 - Removed calls to cond_resched(). The lower-level driver must manage
   any scheduler issues.
 - Removed the request_cancel flag and handling from the class driver
   including the fpga_image_prog_transition() function.
 - The cancel system call now directly calls the cancel() op of the
   lower-level driver. 
 - Changed the cancel() op to return void.

Changelog v15 -> v16:
 - Remove previous patch #5: fpga: image-load: create status sysfs node
 - Change device name from "fpga_image%d" to "fpga_image_load%d"
 - Shift from "Driver" terminology to "Framework" in comments and
   documentation
 - Some cleanup of documentation for the FPGA_IMAGE_LOAD_WRITE IOCTL.
 - Rename lops to ops for structure member and local variables
 - Change the write_blk() definition to pass in *blk_size (a pointer to
   a default block size of WRITE_BLOCK_SIZE=0x4000) and max_size (the
   the maximum block size to stay within the limit of the data buffer).
   The write_blk() op may use the default *blk_size or modify it to a
   more optimal number for the given device, subject to the max_size limit.
 - All enum values for progress and errors are changed to macros, because
   they are included in the uapi header. This is done to maintain consistency
   amongst the DFL related IOCTL header files. All references to the enum
   types are changed to u32.
 - Bail out early in fpga_image_do_load() if imgld->driver_unload is true.
 - Add a call to cond_resched() in the write_blk() loop to ensure that
   we yield to higher priority tasks during long data transfers.
 - Switch to the system_unbound_wq to enable calls to cond_resched().
 - Switch from test_and_set_bit() to atomic_cmpxchg() to manage
   imgld->opened.
 - Change fpga_image_load_release() cancel an ongoing image upload when
   possible and to block when cancellation is not possible.
 - Remove the completion object, imgld->update_done, in favor of calling
   flush_work(&imgld->work);

Changelog v14 -> v15:
 - Changed the driver name from FPGA Security Manager to FPGA Image Load
 - Changed file, symbol, and config names to reflect the new driver name
 - Rewrote documentation.
 - Removed all sysfs files except for "status", which has moved to the
   parent directory.
 - Implemented FPGA_IMAGE_LOAD_WRITE, FPGA_IMAGE_LOAD_STATUS, and
   FPGA_IMAGE_LOAD_CANCEL IOCTLs to initiate, monitor, and cancel image
   uploads.
 - Fixed some error return values in fpga_image_load_register()
 - Added an eventfd which is signalled upon completion of an image load.
 - Minor changes to locking to accommodate the FPGA_IMAGE_LOAD_STATUS IOCTL
 - Removed signed-off/reviewed-by tags. Please resend as appropriate.

Changelog v13 -> v14:
 - Dropped the patch: fpga: sec-mgr: expose hardware error info
 - Updated copyrights to 2021
 - Updated ABI documentation date and kernel version
 - Removed the name sysfs entry from the class driver
 - Use xa_alloc() instead of ida_simple_get()
 - Rename dev to parent for parent devices
 - Remove fpga_sec_mgr_create(), devm_fpga_sec_mgr_create(), and
   fpga_sec_mgr_free() functions and update the fpga_sec_mgr_register()
   function to both create and register a new security manager.
 - Populate the fpga_sec_mgr_dev_release() function.
 - Added MAINTAINERS reference for
   Documentation/ABI/testing/sysfs-class-fpga-sec-mgr

Changelog v12 -> v13:
  - Change "if (count == 0 || " to "if (!count || "
  - Improve error message: "Attempt to register without all required ops\n"
  - Change set_error() to fpga_sec_set_error()
  - Change set_hw_errinfo() to fpga_sec_set_hw_errinfo()

Changelog v11 -> v12:
  - Updated Date and KernelVersion fields in ABI documentation
  - Removed size parameter from write_blk() op - it is now up to
    the lower-level driver to determine the appropriate size and
    to update smgr->remaining_size accordingly.
  - Changed syntax of sec_mgr_prog_str[] and sec_mgr_err_str array definitions
    from:
	"idle",			/* FPGA_SEC_PROG_IDLE */
    to:
	[FPGA_SEC_PROG_IDLE]	    = "idle",

Changelog v10 -> v11:
  - Fixed a spelling error in a comment
  - Initialize smgr->err_code and smgr->progress explicitly in
    fpga_sec_mgr_create() instead of accepting the default 0 value.

Changelog v9 -> v10:
  - Rebased to 5.12-rc2 next
  - Updated Date and KernelVersion in ABI documentation

Changelog v8 -> v9:
  - Rebased patches for 5.11-rc2
  - Updated Date and KernelVersion in ABI documentation

Changelog v7 -> v8:
  - Fixed grammatical error in Documentation/fpga/fpga-sec-mgr.rst

Changelog v6 -> v7:
  - Changed dates in documentation file to December 2020
  - Changed filename_store() to use kmemdup_nul() instead of
    kstrndup() and changed the count to not assume a line-return.

Changelog v5 -> v6:
  - Removed sysfs support and documentation for the display of the
    flash count, root entry hashes, and code-signing-key cancellation
    vectors from the class driver. This information can vary by device
    and will instead be displayed by the device-specific parent driver.

Changelog v4 -> v5:
  - Added the devm_fpga_sec_mgr_unregister() function, following recent
    changes to the fpga_manager() implementation.
  - Changed most of the *_show() functions to use sysfs_emit()
    instead of sprintf(
  - When checking the return values for functions of type enum
    fpga_sec_err err_code, test for FPGA_SEC_ERR_NONE instead of 0

Changelog v3 -> v4:
  - This driver is generic enough that it could be used for non Intel
    FPGA devices. Changed from "Intel FPGA Security Manager" to FPGA
    Security Manager" and removed unnecessary references to "Intel".
  - Changed: iops -> sops, imgr -> smgr, IFPGA_ -> FPGA_, ifpga_ to fpga_
    Note that this also affects some filenames.

Changelog v2 -> v3:
  - Use dev_err() to report invalid progress in sec_progress()
  - Use dev_err() to report invalid error code in sec_error()
  - Modified sysfs handler check in check_sysfs_handler() to make
    it more readable.
  - Removed unnecessary "goto done"
  - Added a comment to explain imgr->driver_unload in
    ifpga_sec_mgr_unregister()

Changelog v1 -> v2:
  - Separated out the MAX10 BMC Security Engine to be submitted in
    a separate patch-set.
  - Bumped documentation dates and versions
  - Split ifpga_sec_mgr_register() into create() and register() functions
  - Added devm_ifpga_sec_mgr_create()
  - Added Documentation/fpga/ifpga-sec-mgr.rst 
  - Changed progress state "read_file" to "reading"
  - Added sec_error() function (similar to sec_progress())
  - Removed references to bmc_flash_count & smbus_flash_count (not supported)
  - Removed typedefs for imgr ops
  - Removed explicit value assignments in enums
  - Other minor code cleanup per review comments 

Russ Weight (5):
  fpga: image-load: fpga image load framework
  fpga: image-load: enable image uploads
  fpga: image-load: signal eventfd when complete
  fpga: image-load: add status ioctl
  fpga: image-load: enable cancel of image upload

 Documentation/fpga/fpga-image-load.rst |  50 +++
 Documentation/fpga/index.rst           |   1 +
 MAINTAINERS                            |   9 +
 drivers/fpga/Kconfig                   |  10 +
 drivers/fpga/Makefile                  |   3 +
 drivers/fpga/fpga-image-load.c         | 416 +++++++++++++++++++++++++
 include/linux/fpga/fpga-image-load.h   |  69 ++++
 include/uapi/linux/fpga-image-load.h   |  74 +++++
 8 files changed, 632 insertions(+)
 create mode 100644 Documentation/fpga/fpga-image-load.rst
 create mode 100644 drivers/fpga/fpga-image-load.c
 create mode 100644 include/linux/fpga/fpga-image-load.h
 create mode 100644 include/uapi/linux/fpga-image-load.h

-- 
2.25.1

[PATCH v17 1/5] fpga: image-load: fpga image load framework

[Russ Weight]

The FPGA Image Load framework provides an API to transfer update
files to an FPGA device. Image files are self-describing. They could
contain FPGA images, BMC images, Root Entry Hashes, or other device
specific files. It is up to the lower level device driver and the
target device to authenticate and disposition the file data.

Signed-off-by: Russ Weight <russell.h.weight@intel.com>
---
v17:
 - No change
v16:
 - Shift from "Driver" terminology to "Framework" in comments and
   documentation
 - Rename lops to ops for structure member and local variables
 - Change device name from "fpga_image%d" to "fpga_image_load%d"
v15:
 - Compare to previous patch:
     [PATCH v14 1/6] fpga: sec-mgr: fpga security manager class driver 
 - Changed file, symbol, and config names to reflect the new driver name
 - Rewrote documentation. The documentation will be added to in later patches.
 - Removed signed-off/reviewed-by tags
v14:
 - Updated copyright to 2021
 - Removed the name sysfs entry
 - Removed MAINTAINERS reference to
   Documentation/ABI/testing/sysfs-class-fpga-sec-mgr
 - Use xa_alloc() instead of ida_simple_get()
 - Rename dev to parent for parent devices
 - Remove fpga_sec_mgr_create(), devm_fpga_sec_mgr_create(), and
   fpga_sec_mgr_free() functions and update the fpga_sec_mgr_register()
   function to both create and register a new security manager.
 - Populate the fpga_sec_mgr_dev_release() function.
v13:
  - No change
v12:
  - Updated Date and KernelVersion fields in ABI documentation
v11:
  - No change
v10:
  - Rebased to 5.12-rc2 next
  - Updated Date and KernelVersion in ABI documentation
v9:
  - Updated Date and KernelVersion in ABI documentation
v8:
  - Fixed grammatical error in Documentation/fpga/fpga-sec-mgr.rst
v7:
  - Changed Date in documentation file to December 2020
v6:
  - Removed sysfs support and documentation for the display of the
    flash count, root entry hashes, and code-signing-key cancelation
    vectors.
v5:
  - Added the devm_fpga_sec_mgr_unregister() function, following recent
    changes to the fpga_manager() implementation.
  - Changed some *_show() functions to use sysfs_emit() instead of sprintf(
v4:
  - Changed from "Intel FPGA Security Manager" to FPGA Security Manager"
    and removed unnecessary references to "Intel".
  - Changed: iops -> sops, imgr -> smgr, IFPGA_ -> FPGA_, ifpga_ to fpga_
v3:
  - Modified sysfs handler check in check_sysfs_handler() to make
    it more readable.
v2:
  - Bumped documentation dates and versions
  - Added Documentation/fpga/ifpga-sec-mgr.rst
  - Removed references to bmc_flash_count & smbus_flash_count (not supported)
  - Split ifpga_sec_mgr_register() into create() and register() functions
  - Added devm_ifpga_sec_mgr_create()
  - Removed typedefs for imgr ops
---
 Documentation/fpga/fpga-image-load.rst |  10 ++
 Documentation/fpga/index.rst           |   1 +
 MAINTAINERS                            |   8 ++
 drivers/fpga/Kconfig                   |  10 ++
 drivers/fpga/Makefile                  |   3 +
 drivers/fpga/fpga-image-load.c         | 125 +++++++++++++++++++++++++
 include/linux/fpga/fpga-image-load.h   |  35 +++++++
 7 files changed, 192 insertions(+)
 create mode 100644 Documentation/fpga/fpga-image-load.rst
 create mode 100644 drivers/fpga/fpga-image-load.c
 create mode 100644 include/linux/fpga/fpga-image-load.h

diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
new file mode 100644
index 000000000000..dda9283ef1c7
--- /dev/null
+++ b/Documentation/fpga/fpga-image-load.rst
@@ -0,0 +1,10 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+FPGA Image Load Framework
+=========================
+
+The FPGA Image Load framework provides a common API for user-space
+tools to manage image uploads to FPGA devices. Device drivers that
+instantiate the FPGA Image Load framework will interact with the
+target device to transfer and authenticate the image data.
diff --git a/Documentation/fpga/index.rst b/Documentation/fpga/index.rst
index f80f95667ca2..85d25fb22c08 100644
--- a/Documentation/fpga/index.rst
+++ b/Documentation/fpga/index.rst
@@ -8,6 +8,7 @@ fpga
     :maxdepth: 1
 
     dfl
+    fpga-image-load
 
 .. only::  subproject and html
 
diff --git a/MAINTAINERS b/MAINTAINERS
index 437247390737..a99622eebbff 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7408,6 +7408,14 @@ F:	Documentation/fpga/
 F:	drivers/fpga/
 F:	include/linux/fpga/
 
+FPGA IMAGE UPLOAD DRIVERS
+M:	Russ Weight <russell.h.weight@intel.com>
+L:	linux-fpga@vger.kernel.org
+S:	Maintained
+F:	Documentation/fpga/fpga-image-load.rst
+F:	drivers/fpga/fpga-image-load.c
+F:	include/linux/fpga/fpga-image-load.h
+
 FPU EMULATOR
 M:	Bill Metzenthen <billm@melbpc.org.au>
 S:	Maintained
diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
index 991b3f361ec9..fbd580121661 100644
--- a/drivers/fpga/Kconfig
+++ b/drivers/fpga/Kconfig
@@ -243,4 +243,14 @@ config FPGA_MGR_VERSAL_FPGA
 	  configure the programmable logic(PL).
 
 	  To compile this as a module, choose M here.
+
+config FPGA_IMAGE_LOAD
+	tristate "FPGA Image Load Framework"
+	help
+	  The FPGA Image Load framework presents a common user API for
+	  uploading an image file to an FPGA device. The image file is
+	  expected to be self-describing. It is up to the lower level
+	  device driver and/or the device itself to authenticate and
+	  disposition the image data.
+
 endif # FPGA
diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile
index 0bff783d1b61..adf228ee4f5e 100644
--- a/drivers/fpga/Makefile
+++ b/drivers/fpga/Makefile
@@ -22,6 +22,9 @@ obj-$(CONFIG_FPGA_MGR_VERSAL_FPGA)      += versal-fpga.o
 obj-$(CONFIG_ALTERA_PR_IP_CORE)         += altera-pr-ip-core.o
 obj-$(CONFIG_ALTERA_PR_IP_CORE_PLAT)    += altera-pr-ip-core-plat.o
 
+# FPGA Image Load Framework
+obj-$(CONFIG_FPGA_IMAGE_LOAD)		+= fpga-image-load.o
+
 # FPGA Bridge Drivers
 obj-$(CONFIG_FPGA_BRIDGE)		+= fpga-bridge.o
 obj-$(CONFIG_SOCFPGA_FPGA_BRIDGE)	+= altera-hps2fpga.o altera-fpga2sdram.o
diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
new file mode 100644
index 000000000000..799d18444f7c
--- /dev/null
+++ b/drivers/fpga/fpga-image-load.c
@@ -0,0 +1,125 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * FPGA Image Load Framework
+ *
+ * Copyright (C) 2019-2021 Intel Corporation, Inc.
+ */
+
+#include <linux/fpga/fpga-image-load.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#define IMAGE_LOAD_XA_LIMIT	XA_LIMIT(0, INT_MAX)
+static DEFINE_XARRAY_ALLOC(fpga_image_load_xa);
+
+static struct class *fpga_image_load_class;
+
+#define to_image_load(d) container_of(d, struct fpga_image_load, dev)
+
+/**
+ * fpga_image_load_register - create and register an FPGA Image Load Device
+ *
+ * @parent: fpga image load device from pdev
+ * @ops:   pointer to a structure of image load callback functions
+ * @priv:   fpga image load private data
+ *
+ * Returns a struct fpga_image_load pointer on success, or ERR_PTR() on
+ * error. The caller of this function is responsible for calling
+ * fpga_image_load_unregister().
+ */
+struct fpga_image_load *
+fpga_image_load_register(struct device *parent,
+			 const struct fpga_image_load_ops *ops, void *priv)
+{
+	struct fpga_image_load *imgld;
+	int ret;
+
+	imgld = kzalloc(sizeof(*imgld), GFP_KERNEL);
+	if (!imgld)
+		return ERR_PTR(-ENOMEM);
+
+	ret = xa_alloc(&fpga_image_load_xa, &imgld->dev.id, imgld, IMAGE_LOAD_XA_LIMIT,
+		       GFP_KERNEL);
+	if (ret)
+		goto error_kfree;
+
+	mutex_init(&imgld->lock);
+
+	imgld->priv = priv;
+	imgld->ops = ops;
+
+	imgld->dev.class = fpga_image_load_class;
+	imgld->dev.parent = parent;
+
+	ret = dev_set_name(&imgld->dev, "fpga_image_load%d", imgld->dev.id);
+	if (ret) {
+		dev_err(parent, "Failed to set device name: fpga_image_load%d\n",
+			imgld->dev.id);
+		goto error_device;
+	}
+
+	ret = device_register(&imgld->dev);
+	if (ret) {
+		put_device(&imgld->dev);
+		return ERR_PTR(ret);
+	}
+
+	return imgld;
+
+error_device:
+	xa_erase(&fpga_image_load_xa, imgld->dev.id);
+
+error_kfree:
+	kfree(imgld);
+
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(fpga_image_load_register);
+
+/**
+ * fpga_image_load_unregister - unregister an FPGA image load device
+ *
+ * @imgld: pointer to struct fpga_image_load
+ *
+ * This function is intended for use in the parent driver's remove()
+ * function.
+ */
+void fpga_image_load_unregister(struct fpga_image_load *imgld)
+{
+	device_unregister(&imgld->dev);
+}
+EXPORT_SYMBOL_GPL(fpga_image_load_unregister);
+
+static void fpga_image_load_dev_release(struct device *dev)
+{
+	struct fpga_image_load *imgld = to_image_load(dev);
+
+	xa_erase(&fpga_image_load_xa, imgld->dev.id);
+	kfree(imgld);
+}
+
+static int __init fpga_image_load_class_init(void)
+{
+	pr_info("FPGA Image Load Framework\n");
+
+	fpga_image_load_class = class_create(THIS_MODULE, "fpga_image_load");
+	if (IS_ERR(fpga_image_load_class))
+		return PTR_ERR(fpga_image_load_class);
+
+	fpga_image_load_class->dev_release = fpga_image_load_dev_release;
+
+	return 0;
+}
+
+static void __exit fpga_image_load_class_exit(void)
+{
+	class_destroy(fpga_image_load_class);
+	WARN_ON(!xa_empty(&fpga_image_load_xa));
+}
+
+MODULE_DESCRIPTION("FPGA Image Load Framework");
+MODULE_LICENSE("GPL v2");
+
+subsys_initcall(fpga_image_load_class_init);
+module_exit(fpga_image_load_class_exit)
diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
new file mode 100644
index 000000000000..8b051c82ef5f
--- /dev/null
+++ b/include/linux/fpga/fpga-image-load.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Header file for FPGA Image Load Framework
+ *
+ * Copyright (C) 2019-2021 Intel Corporation, Inc.
+ */
+#ifndef _LINUX_FPGA_IMAGE_LOAD_H
+#define _LINUX_FPGA_IMAGE_LOAD_H
+
+#include <linux/device.h>
+#include <linux/mutex.h>
+#include <linux/types.h>
+
+struct fpga_image_load;
+
+/**
+ * struct fpga_image_load_ops - device specific operations
+ */
+struct fpga_image_load_ops {
+};
+
+struct fpga_image_load {
+	struct device dev;
+	const struct fpga_image_load_ops *ops;
+	struct mutex lock;		/* protect data structure contents */
+	void *priv;
+};
+
+struct fpga_image_load *
+fpga_image_load_register(struct device *dev,
+			 const struct fpga_image_load_ops *ops, void *priv);
+
+void fpga_image_load_unregister(struct fpga_image_load *imgld);
+
+#endif
-- 
2.25.1

[PATCH v17 2/5] fpga: image-load: enable image uploads

[Russ Weight]

Extend the FPGA Image Load framework to include IOCTL support
(FPGA_IMAGE_LOAD_WRITE) for initiating an upload of an image to a device.
The IOCTL will return immediately, and the update will begin in the
context of a kernel worker thread.

Signed-off-by: Russ Weight <russell.h.weight@intel.com>
---
v17:
 - Minor documentation cleanup and references to cancellation are moved
   to a later patch.
 - We no longer send a "just in case" cancel on HW errors. The cleanup()
   op can and should be used to manage the HW state on an error.
 - Removed the FPGA_IMAGE_ERR_NONE macro in favor of returning zero. The
   constant name made sense when we were using enum data types, but is no
   longer necessary.
 - Changed the syntax of the write_blk() function and renamed it to write().
   It now returns a positive size or a negative error number.
 - Changed the prepare() and write() ops to pass in elements of the
   fpga_image_load data structure instead of having the op functions access
   them directly via imgld.
 - Removed the default block size. The lower-level driver must choose a
   block size.
 - Changed work queue from system_unbound_wq to system_long_wq.
 - Removed calls to cond_resched(). The lower-level driver must manage
   any scheduler issues.
 - Use the cdev_set_parent() macro.
v16:
 - Shift from "Driver" terminology to "Framework" in comments and
   documentation
 - Rename lops to ops for structure member and local variables
 - Change the write_blk() definition to pass in *blk_size (a pointer to
   a default block size of WRITE_BLOCK_SIZE=0x4000) and max_size (the
   the maximum block size to stay within the limit of the data buffer).
   The write_blk() op may use the default *blk_size or modify it to a
   more optimal number for the given device, subject to the max_size limit.
 - All enum values for progress and errors are changed to macros, because
   they are included in the uapi header. This is done to maintain consistency
   amongst the DFL related IOCTL header files. All references to the enum
   types are changed to u32.
 - Bail out early in fpga_image_do_load() if imgld->driver_unload is true.
 - Add a call to cond_resched() in the write_blk() loop to ensure that
   we yield to higher priority tasks during long data transfers.
 - Switch to the system_unbound_wq to enable calls to cond_resched().
 - Switch from test_and_set_bit() to atomic_cmpxchg() to manage
   imgld->opened.
 - Change fpga_image_load_release() to block until the image upload is
   complete.
 - Remove the completion object, imgld->update_done, in favor of calling
   flush_work(&imgld->work);
v15:
 - Compare to previous patch:
     [PATCH v14 2/6] fpga: sec-mgr: enable secure updates
 - Changed file, symbol, and config names to reflect the new driver name
 - Removed update/filename sysfs file and added the FPGA_IMAGE_LOAD_WRITE
   IOCTL for writing the image data to the FPGA card. The driver no longer
   uses the request_firmware framework.
 - Fixed some error return values in fpga_image_load_register()
 - Removed signed-off/reviewed-by tags
v14:
 - Added MAINTAINERS reference for
   Documentation/ABI/testing/sysfs-class-fpga-sec-mgr
 - Updated ABI documentation date and kernel version
 - Updated copyright to 2021
v13:
  - Change "if (count == 0 || " to "if (!count || "
  - Improve error message: "Attempt to register without all required ops\n"
v12:
  - Updated Date and KernelVersion fields in ABI documentation
  - Removed size parameter from write_blk() op - it is now up to
    the lower-level driver to determine the appropriate size and
    to update smgr->remaining_size accordingly.
v11:
  - Fixed a spelling error in a comment
  - Initialize smgr->err_code and smgr->progress explicitly in
    fpga_sec_mgr_create() instead of accepting the default 0 value.
v10:
  - Rebased to 5.12-rc2 next
  - Updated Date and KernelVersion in ABI documentation
v9:
  - Updated Date and KernelVersion in ABI documentation
v8:
  - No change
v7:
  - Changed Date in documentation file to December 2020
  - Changed filename_store() to use kmemdup_nul() instead of
    kstrndup() and changed the count to not assume a line-return.
v6:
  - Changed "security update" to "secure update" in commit message
v5:
  - When checking the return values for functions of type enum
    fpga_sec_err err_code, test for FPGA_SEC_ERR_NONE instead of 0
v4:
  - Changed from "Intel FPGA Security Manager" to FPGA Security Manager"
    and removed unnecessary references to "Intel".
  - Changed: iops -> sops, imgr -> smgr, IFPGA_ -> FPGA_, ifpga_ to fpga_
v3:
  - Removed unnecessary "goto done"
  - Added a comment to explain imgr->driver_unload in
    ifpga_sec_mgr_unregister()
v2:
  - Bumped documentation date and version
  - Removed explicit value assignments in enums
  - Other minor code cleanup per review comments
---
 Documentation/fpga/fpga-image-load.rst |  25 ++-
 MAINTAINERS                            |   1 +
 drivers/fpga/fpga-image-load.c         | 217 ++++++++++++++++++++++++-
 include/linux/fpga/fpga-image-load.h   |  27 +++
 include/uapi/linux/fpga-image-load.h   |  53 ++++++
 5 files changed, 321 insertions(+), 2 deletions(-)
 create mode 100644 include/uapi/linux/fpga-image-load.h

diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
index dda9283ef1c7..f6b0e1624a05 100644
--- a/Documentation/fpga/fpga-image-load.rst
+++ b/Documentation/fpga/fpga-image-load.rst
@@ -7,4 +7,27 @@ FPGA Image Load Framework
 The FPGA Image Load framework provides a common API for user-space
 tools to manage image uploads to FPGA devices. Device drivers that
 instantiate the FPGA Image Load framework will interact with the
-target device to transfer and authenticate the image data.
+target device to transfer and authenticate the image data. Image uploads
+are processed in the context of a kernel worker thread.
+
+User API
+========
+
+open
+----
+
+An fpga_image_load device is opened exclusively to control an image upload.
+The device must remain open throughout the duration of the image upload.
+An attempt to close the device while an upload is in progress will block
+until the image upload is complete.
+
+ioctl
+-----
+
+FPGA_IMAGE_LOAD_WRITE:
+
+Start an image upload with the provided image buffer. This IOCTL returns
+immediately after starting a kernel worker thread to process the image
+upload which could take as long as 40 minutes depending on the actual
+device being updated. This is an exclusive operation; an attempt to start
+concurrent image uploads for the same device will fail with EBUSY.
diff --git a/MAINTAINERS b/MAINTAINERS
index a99622eebbff..e3b5c555ecbd 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7415,6 +7415,7 @@ S:	Maintained
 F:	Documentation/fpga/fpga-image-load.rst
 F:	drivers/fpga/fpga-image-load.c
 F:	include/linux/fpga/fpga-image-load.h
+F:	include/uapi/linux/fpga-image-load.h
 
 FPU EMULATOR
 M:	Bill Metzenthen <billm@melbpc.org.au>
diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
index 799d18444f7c..4c2571e3b26f 100644
--- a/drivers/fpga/fpga-image-load.c
+++ b/drivers/fpga/fpga-image-load.c
@@ -5,18 +5,190 @@
  * Copyright (C) 2019-2021 Intel Corporation, Inc.
  */
 
+#include <linux/delay.h>
 #include <linux/fpga/fpga-image-load.h>
+#include <linux/fs.h>
+#include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/slab.h>
+#include <linux/uaccess.h>
 #include <linux/vmalloc.h>
 
 #define IMAGE_LOAD_XA_LIMIT	XA_LIMIT(0, INT_MAX)
 static DEFINE_XARRAY_ALLOC(fpga_image_load_xa);
 
 static struct class *fpga_image_load_class;
+static dev_t fpga_image_devt;
 
 #define to_image_load(d) container_of(d, struct fpga_image_load, dev)
 
+static void fpga_image_prog_complete(struct fpga_image_load *imgld)
+{
+	mutex_lock(&imgld->lock);
+	imgld->progress = FPGA_IMAGE_PROG_IDLE;
+	mutex_unlock(&imgld->lock);
+}
+
+static void fpga_image_do_load(struct work_struct *work)
+{
+	struct fpga_image_load *imgld;
+	s32 ret, offset = 0;
+
+	imgld = container_of(work, struct fpga_image_load, work);
+
+	if (imgld->driver_unload) {
+		imgld->err_code = FPGA_IMAGE_ERR_CANCELED;
+		goto idle_exit;
+	}
+
+	get_device(&imgld->dev);
+	if (!try_module_get(imgld->dev.parent->driver->owner)) {
+		imgld->err_code = FPGA_IMAGE_ERR_BUSY;
+		goto putdev_exit;
+	}
+
+	imgld->progress = FPGA_IMAGE_PROG_PREPARING;
+	ret = imgld->ops->prepare(imgld, imgld->data, imgld->remaining_size);
+	if (ret) {
+		imgld->err_code = ret;
+		goto modput_exit;
+	}
+
+	imgld->progress = FPGA_IMAGE_PROG_WRITING;
+	while (imgld->remaining_size) {
+		ret = imgld->ops->write(imgld, imgld->data, offset,
+					imgld->remaining_size);
+		if (ret <= 0) {
+			if (!ret) {
+				dev_warn(&imgld->dev,
+					 "write-op wrote zero data\n");
+				ret = -FPGA_IMAGE_ERR_RW_ERROR;
+			}
+			imgld->err_code = -ret;
+			goto done;
+		}
+
+		imgld->remaining_size -= ret;
+		offset += ret;
+	}
+
+	imgld->progress = FPGA_IMAGE_PROG_PROGRAMMING;
+	ret = imgld->ops->poll_complete(imgld);
+	if (ret)
+		imgld->err_code = ret;
+
+done:
+	if (imgld->ops->cleanup)
+		imgld->ops->cleanup(imgld);
+
+modput_exit:
+	module_put(imgld->dev.parent->driver->owner);
+
+putdev_exit:
+	put_device(&imgld->dev);
+
+idle_exit:
+	/*
+	 * Note: imgld->remaining_size is left unmodified here to provide
+	 * additional information on errors. It will be reinitialized when
+	 * the next image load begins.
+	 */
+	vfree(imgld->data);
+	imgld->data = NULL;
+	fpga_image_prog_complete(imgld);
+}
+
+static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
+				       unsigned long arg)
+{
+	struct fpga_image_write wb;
+	unsigned long minsz;
+	u8 *buf;
+
+	if (imgld->driver_unload || imgld->progress != FPGA_IMAGE_PROG_IDLE)
+		return -EBUSY;
+
+	minsz = offsetofend(struct fpga_image_write, buf);
+	if (copy_from_user(&wb, (void __user *)arg, minsz))
+		return -EFAULT;
+
+	if (wb.flags)
+		return -EINVAL;
+
+	buf = vzalloc(wb.size);
+	if (!buf)
+		return -ENOMEM;
+
+	if (copy_from_user(buf, u64_to_user_ptr(wb.buf), wb.size)) {
+		vfree(buf);
+		return -EFAULT;
+	}
+
+	imgld->data = buf;
+	imgld->remaining_size = wb.size;
+	imgld->err_code = 0;
+	imgld->progress = FPGA_IMAGE_PROG_STARTING;
+	queue_work(system_long_wq, &imgld->work);
+
+	return 0;
+}
+
+static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
+				  unsigned long arg)
+{
+	struct fpga_image_load *imgld = filp->private_data;
+	int ret = -ENOTTY;
+
+	switch (cmd) {
+	case FPGA_IMAGE_LOAD_WRITE:
+		mutex_lock(&imgld->lock);
+		ret = fpga_image_load_ioctl_write(imgld, arg);
+		mutex_unlock(&imgld->lock);
+		break;
+	}
+
+	return ret;
+}
+
+static int fpga_image_load_open(struct inode *inode, struct file *filp)
+{
+	struct fpga_image_load *imgld = container_of(inode->i_cdev,
+						     struct fpga_image_load, cdev);
+
+	if (atomic_cmpxchg(&imgld->opened, 0, 1))
+		return -EBUSY;
+
+	filp->private_data = imgld;
+
+	return 0;
+}
+
+static int fpga_image_load_release(struct inode *inode, struct file *filp)
+{
+	struct fpga_image_load *imgld = filp->private_data;
+
+	mutex_lock(&imgld->lock);
+	if (imgld->progress == FPGA_IMAGE_PROG_IDLE) {
+		mutex_unlock(&imgld->lock);
+		goto close_exit;
+	}
+
+	mutex_unlock(&imgld->lock);
+	flush_work(&imgld->work);
+
+close_exit:
+	atomic_set(&imgld->opened, 0);
+
+	return 0;
+}
+
+static const struct file_operations fpga_image_load_fops = {
+	.owner = THIS_MODULE,
+	.open = fpga_image_load_open,
+	.release = fpga_image_load_release,
+	.unlocked_ioctl = fpga_image_load_ioctl,
+};
+
 /**
  * fpga_image_load_register - create and register an FPGA Image Load Device
  *
@@ -35,6 +207,11 @@ fpga_image_load_register(struct device *parent,
 	struct fpga_image_load *imgld;
 	int ret;
 
+	if (!ops || !ops->prepare || !ops->write || !ops->poll_complete) {
+		dev_err(parent, "Attempt to register without all required ops\n");
+		return ERR_PTR(-ENOMEM);
+	}
+
 	imgld = kzalloc(sizeof(*imgld), GFP_KERNEL);
 	if (!imgld)
 		return ERR_PTR(-ENOMEM);
@@ -48,9 +225,13 @@ fpga_image_load_register(struct device *parent,
 
 	imgld->priv = priv;
 	imgld->ops = ops;
+	imgld->err_code = 0;
+	imgld->progress = FPGA_IMAGE_PROG_IDLE;
+	INIT_WORK(&imgld->work, fpga_image_do_load);
 
 	imgld->dev.class = fpga_image_load_class;
 	imgld->dev.parent = parent;
+	imgld->dev.devt = MKDEV(MAJOR(fpga_image_devt), imgld->dev.id);
 
 	ret = dev_set_name(&imgld->dev, "fpga_image_load%d", imgld->dev.id);
 	if (ret) {
@@ -65,6 +246,16 @@ fpga_image_load_register(struct device *parent,
 		return ERR_PTR(ret);
 	}
 
+	cdev_init(&imgld->cdev, &fpga_image_load_fops);
+	imgld->cdev.owner = parent->driver->owner;
+	cdev_set_parent(&imgld->cdev, &imgld->dev.kobj);
+
+	ret = cdev_add(&imgld->cdev, imgld->dev.devt, 1);
+	if (ret) {
+		put_device(&imgld->dev);
+		return ERR_PTR(ret);
+	}
+
 	return imgld;
 
 error_device:
@@ -83,10 +274,23 @@ EXPORT_SYMBOL_GPL(fpga_image_load_register);
  * @imgld: pointer to struct fpga_image_load
  *
  * This function is intended for use in the parent driver's remove()
- * function.
+ * function. The driver_unload flag prevents new updates from starting
+ * once the unregister process has begun.
  */
 void fpga_image_load_unregister(struct fpga_image_load *imgld)
 {
+	mutex_lock(&imgld->lock);
+	imgld->driver_unload = true;
+	if (imgld->progress == FPGA_IMAGE_PROG_IDLE) {
+		mutex_unlock(&imgld->lock);
+		goto unregister;
+	}
+
+	mutex_unlock(&imgld->lock);
+	flush_work(&imgld->work);
+
+unregister:
+	cdev_del(&imgld->cdev);
 	device_unregister(&imgld->dev);
 }
 EXPORT_SYMBOL_GPL(fpga_image_load_unregister);
@@ -101,19 +305,30 @@ static void fpga_image_load_dev_release(struct device *dev)
 
 static int __init fpga_image_load_class_init(void)
 {
+	int ret;
 	pr_info("FPGA Image Load Framework\n");
 
 	fpga_image_load_class = class_create(THIS_MODULE, "fpga_image_load");
 	if (IS_ERR(fpga_image_load_class))
 		return PTR_ERR(fpga_image_load_class);
 
+	ret = alloc_chrdev_region(&fpga_image_devt, 0, MINORMASK,
+				  "fpga_image_load");
+	if (ret)
+		goto exit_destroy_class;
+
 	fpga_image_load_class->dev_release = fpga_image_load_dev_release;
 
 	return 0;
+
+exit_destroy_class:
+	class_destroy(fpga_image_load_class);
+	return ret;
 }
 
 static void __exit fpga_image_load_class_exit(void)
 {
+	unregister_chrdev_region(fpga_image_devt, MINORMASK);
 	class_destroy(fpga_image_load_class);
 	WARN_ON(!xa_empty(&fpga_image_load_xa));
 }
diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
index 8b051c82ef5f..7b445f489644 100644
--- a/include/linux/fpga/fpga-image-load.h
+++ b/include/linux/fpga/fpga-image-load.h
@@ -7,22 +7,49 @@
 #ifndef _LINUX_FPGA_IMAGE_LOAD_H
 #define _LINUX_FPGA_IMAGE_LOAD_H
 
+#include <linux/cdev.h>
 #include <linux/device.h>
 #include <linux/mutex.h>
 #include <linux/types.h>
+#include <uapi/linux/fpga-image-load.h>
 
 struct fpga_image_load;
 
 /**
  * struct fpga_image_load_ops - device specific operations
+ * @prepare:		    Required: Prepare secure update
+ * @write:		    Required: The write() op receives the remaining
+ *			    size to be written and must return the actual
+ *			    size written or a negative error code. The write()
+ *			    op will be called repeatedly until all data is
+ *			    written.
+ * @poll_complete:	    Required: Check for the completion of the
+ *			    HW authentication/programming process.
+ * @cleanup:		    Optional: Complements the prepare()
+ *			    function and is called at the completion
+ *			    of the update, whether success or failure,
+ *			    if the prepare function succeeded.
  */
 struct fpga_image_load_ops {
+	u32 (*prepare)(struct fpga_image_load *imgld, const u8 *data, u32 size);
+	s32 (*write)(struct fpga_image_load *imgld, const u8 *data,
+		     u32 offset, u32 size);
+	u32 (*poll_complete)(struct fpga_image_load *imgld);
+	void (*cleanup)(struct fpga_image_load *imgld);
 };
 
 struct fpga_image_load {
 	struct device dev;
+	struct cdev cdev;
 	const struct fpga_image_load_ops *ops;
 	struct mutex lock;		/* protect data structure contents */
+	atomic_t opened;
+	struct work_struct work;
+	const u8 *data;			/* pointer to update data */
+	u32 remaining_size;		/* size remaining to transfer */
+	u32 progress;
+	u32 err_code;			/* image load error code */
+	bool driver_unload;
 	void *priv;
 };
 
diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
new file mode 100644
index 000000000000..20eae3bb10d8
--- /dev/null
+++ b/include/uapi/linux/fpga-image-load.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Header File for FPGA Image Load User API
+ *
+ * Copyright (C) 2019-2021 Intel Corporation, Inc.
+ *
+ */
+
+#ifndef _UAPI_LINUX_FPGA_IMAGE_LOAD_H
+#define _UAPI_LINUX_FPGA_IMAGE_LOAD_H
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+#define FPGA_IMAGE_LOAD_MAGIC 0xB9
+
+/* Image load progress codes */
+#define FPGA_IMAGE_PROG_IDLE		0
+#define FPGA_IMAGE_PROG_STARTING	1
+#define FPGA_IMAGE_PROG_PREPARING	2
+#define FPGA_IMAGE_PROG_WRITING		3
+#define FPGA_IMAGE_PROG_PROGRAMMING	4
+#define FPGA_IMAGE_PROG_MAX		5
+
+/* Image error progress codes */
+#define FPGA_IMAGE_ERR_HW_ERROR		1
+#define FPGA_IMAGE_ERR_TIMEOUT		2
+#define FPGA_IMAGE_ERR_CANCELED		3
+#define FPGA_IMAGE_ERR_BUSY		4
+#define FPGA_IMAGE_ERR_INVALID_SIZE	5
+#define FPGA_IMAGE_ERR_RW_ERROR		6
+#define FPGA_IMAGE_ERR_WEAROUT		7
+#define FPGA_IMAGE_ERR_MAX		8
+
+/**
+ * FPGA_IMAGE_LOAD_WRITE - _IOW(FPGA_IMAGE_LOAD_MAGIC, 0,
+ *				struct fpga_image_write)
+ *
+ * Upload a data buffer to the target device. The user must provide the
+ * data buffer and size.
+ *
+ * Return: 0 on success, -errno on failure.
+ */
+struct fpga_image_write {
+	/* Input */
+	__u32 flags;		/* Zero for now */
+	__u32 size;		/* Data size (in bytes) to be written */
+	__u64 buf;		/* User space address of source data */
+};
+
+#define FPGA_IMAGE_LOAD_WRITE	_IOW(FPGA_IMAGE_LOAD_MAGIC, 0, struct fpga_image_write)
+
+#endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */
-- 
2.25.1

[PATCH v17 3/5] fpga: image-load: signal eventfd when complete

[Russ Weight]

Amend the FPGA_IMAGE_LOAD_WRITE IOCTL implementation to include an
eventfd file descriptor as a parameter. The eventfd will be triggered
when the image upload completes.

Signed-off-by: Russ Weight <russell.h.weight@intel.com>
---
v17:
 - More documentation cleanup.
v16:
 - Some cleanup of documentation for the FPGA_IMAGE_LOAD_WRITE IOCTL.
v15:
 - This patch is new to the patch-set, and adds an eventfd to the
   FPGA_IMAGE_LOAD_WRITE IOCTL. The eventfd is signalled upon completion
   of an update.
---
 Documentation/fpga/fpga-image-load.rst |  8 +++++---
 drivers/fpga/fpga-image-load.c         | 22 ++++++++++++++++++++--
 include/linux/fpga/fpga-image-load.h   |  2 ++
 include/uapi/linux/fpga-image-load.h   |  3 ++-
 4 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
index f6b0e1624a05..487b5466f67c 100644
--- a/Documentation/fpga/fpga-image-load.rst
+++ b/Documentation/fpga/fpga-image-load.rst
@@ -28,6 +28,8 @@ FPGA_IMAGE_LOAD_WRITE:
 
 Start an image upload with the provided image buffer. This IOCTL returns
 immediately after starting a kernel worker thread to process the image
-upload which could take as long as 40 minutes depending on the actual
-device being updated. This is an exclusive operation; an attempt to start
-concurrent image uploads for the same device will fail with EBUSY.
+upload which could take as long as 40 minutes depending on the actual device
+being updated. This is an exclusive operation; an attempt to start
+concurrent image uploads for the same device will fail with EBUSY. An
+eventfd file descriptor parameter is provided to this IOCTL. It will be
+signalled at the completion of the image upload.
diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
index 4c2571e3b26f..f04dfc71c190 100644
--- a/drivers/fpga/fpga-image-load.c
+++ b/drivers/fpga/fpga-image-load.c
@@ -26,6 +26,7 @@ static void fpga_image_prog_complete(struct fpga_image_load *imgld)
 {
 	mutex_lock(&imgld->lock);
 	imgld->progress = FPGA_IMAGE_PROG_IDLE;
+	eventfd_signal(imgld->finished, 1);
 	mutex_unlock(&imgld->lock);
 }
 
@@ -96,6 +97,8 @@ static void fpga_image_do_load(struct work_struct *work)
 	vfree(imgld->data);
 	imgld->data = NULL;
 	fpga_image_prog_complete(imgld);
+	eventfd_ctx_put(imgld->finished);
+	imgld->finished = NULL;
 }
 
 static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
@@ -103,6 +106,7 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
 {
 	struct fpga_image_write wb;
 	unsigned long minsz;
+	int ret;
 	u8 *buf;
 
 	if (imgld->driver_unload || imgld->progress != FPGA_IMAGE_PROG_IDLE)
@@ -115,13 +119,23 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
 	if (wb.flags)
 		return -EINVAL;
 
+	if (wb.evtfd < 0)
+		return -EINVAL;
+
 	buf = vzalloc(wb.size);
 	if (!buf)
 		return -ENOMEM;
 
 	if (copy_from_user(buf, u64_to_user_ptr(wb.buf), wb.size)) {
-		vfree(buf);
-		return -EFAULT;
+		ret = -EFAULT;
+		goto exit_free;
+	}
+
+	imgld->finished = eventfd_ctx_fdget(wb.evtfd);
+	if (IS_ERR(imgld->finished)) {
+		ret = PTR_ERR(imgld->finished);
+		imgld->finished = NULL;
+		goto exit_free;
 	}
 
 	imgld->data = buf;
@@ -131,6 +145,10 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
 	queue_work(system_long_wq, &imgld->work);
 
 	return 0;
+
+exit_free:
+	vfree(buf);
+	return ret;
 }
 
 static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
index 7b445f489644..77b3c91ce073 100644
--- a/include/linux/fpga/fpga-image-load.h
+++ b/include/linux/fpga/fpga-image-load.h
@@ -9,6 +9,7 @@
 
 #include <linux/cdev.h>
 #include <linux/device.h>
+#include <linux/eventfd.h>
 #include <linux/mutex.h>
 #include <linux/types.h>
 #include <uapi/linux/fpga-image-load.h>
@@ -50,6 +51,7 @@ struct fpga_image_load {
 	u32 progress;
 	u32 err_code;			/* image load error code */
 	bool driver_unload;
+	struct eventfd_ctx *finished;
 	void *priv;
 };
 
diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
index 20eae3bb10d8..8d2d3db92e87 100644
--- a/include/uapi/linux/fpga-image-load.h
+++ b/include/uapi/linux/fpga-image-load.h
@@ -37,7 +37,7 @@
  *				struct fpga_image_write)
  *
  * Upload a data buffer to the target device. The user must provide the
- * data buffer and size.
+ * data buffer, size, and an eventfd file descriptor.
  *
  * Return: 0 on success, -errno on failure.
  */
@@ -45,6 +45,7 @@ struct fpga_image_write {
 	/* Input */
 	__u32 flags;		/* Zero for now */
 	__u32 size;		/* Data size (in bytes) to be written */
+	__s32 evtfd;		/* File descriptor for completion signal */
 	__u64 buf;		/* User space address of source data */
 };
 
-- 
2.25.1

[PATCH v17 4/5] fpga: image-load: add status ioctl

[Russ Weight]

Extend the FPGA Image Load framework to include an FPGA_IMAGE_LOAD_STATUS
IOCTL that can be used to monitor the progress of an ongoing image upload.
The status returned includes how much data remains to be transferred, the
progress of the image upload, and error information in the case of a
failure.

Signed-off-by: Russ Weight <russell.h.weight@intel.com>
---
v17:
 - Rebased for changes to earlier patches.
v16:
 - Minor changes to adapt in changes in prevoius patches.
v15:
 - This patch is new to the patchset and provides an FPGA_IMAGE_LOAD_STATUS
   IOCTL to return the current values for: remaining_size, progress,
   err_progress, and err_code.
 - This patch has elements of the following three patches from the previous
   patch-set:
     [PATCH v14 3/6] fpga: sec-mgr: expose sec-mgr update status
     [PATCH v14 4/6] fpga: sec-mgr: expose sec-mgr update errors
     [PATCH v14 5/6] fpga: sec-mgr: expose sec-mgr update size
 - Changed file, symbol, and config names to reflect the new driver name
 - There are some minor changes to locking to enable this ioctl to return
   coherent data.
---
 Documentation/fpga/fpga-image-load.rst |  6 +++
 drivers/fpga/fpga-image-load.c         | 58 +++++++++++++++++++++-----
 include/linux/fpga/fpga-image-load.h   |  1 +
 include/uapi/linux/fpga-image-load.h   | 18 ++++++++
 4 files changed, 73 insertions(+), 10 deletions(-)

diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
index 487b5466f67c..f64f5ee473b8 100644
--- a/Documentation/fpga/fpga-image-load.rst
+++ b/Documentation/fpga/fpga-image-load.rst
@@ -33,3 +33,9 @@ being updated. This is an exclusive operation; an attempt to start
 concurrent image uploads for the same device will fail with EBUSY. An
 eventfd file descriptor parameter is provided to this IOCTL. It will be
 signalled at the completion of the image upload.
+
+FPGA_IMAGE_LOAD_STATUS:
+
+Collect status for an on-going image upload. The status returned includes
+how much data remains to be transferred, the progress of the image upload,
+and error information in the case of a failure.
diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
index f04dfc71c190..58373b9e8c02 100644
--- a/drivers/fpga/fpga-image-load.c
+++ b/drivers/fpga/fpga-image-load.c
@@ -22,6 +22,22 @@ static dev_t fpga_image_devt;
 
 #define to_image_load(d) container_of(d, struct fpga_image_load, dev)
 
+static void fpga_image_update_progress(struct fpga_image_load *imgld,
+				       u32 new_progress)
+{
+	mutex_lock(&imgld->lock);
+	imgld->progress = new_progress;
+	mutex_unlock(&imgld->lock);
+}
+
+static void fpga_image_set_error(struct fpga_image_load *imgld, u32 err_code)
+{
+	mutex_lock(&imgld->lock);
+	imgld->err_progress = imgld->progress;
+	imgld->err_code = err_code;
+	mutex_unlock(&imgld->lock);
+}
+
 static void fpga_image_prog_complete(struct fpga_image_load *imgld)
 {
 	mutex_lock(&imgld->lock);
@@ -38,24 +54,24 @@ static void fpga_image_do_load(struct work_struct *work)
 	imgld = container_of(work, struct fpga_image_load, work);
 
 	if (imgld->driver_unload) {
-		imgld->err_code = FPGA_IMAGE_ERR_CANCELED;
+		fpga_image_set_error(imgld, FPGA_IMAGE_ERR_CANCELED);
 		goto idle_exit;
 	}
 
 	get_device(&imgld->dev);
 	if (!try_module_get(imgld->dev.parent->driver->owner)) {
-		imgld->err_code = FPGA_IMAGE_ERR_BUSY;
+		fpga_image_set_error(imgld, FPGA_IMAGE_ERR_BUSY);
 		goto putdev_exit;
 	}
 
-	imgld->progress = FPGA_IMAGE_PROG_PREPARING;
+	fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PREPARING);
 	ret = imgld->ops->prepare(imgld, imgld->data, imgld->remaining_size);
 	if (ret) {
-		imgld->err_code = ret;
+		fpga_image_set_error(imgld, ret);
 		goto modput_exit;
 	}
 
-	imgld->progress = FPGA_IMAGE_PROG_WRITING;
+	fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_WRITING);
 	while (imgld->remaining_size) {
 		ret = imgld->ops->write(imgld, imgld->data, offset,
 					imgld->remaining_size);
@@ -65,7 +81,7 @@ static void fpga_image_do_load(struct work_struct *work)
 					 "write-op wrote zero data\n");
 				ret = -FPGA_IMAGE_ERR_RW_ERROR;
 			}
-			imgld->err_code = -ret;
+			fpga_image_set_error(imgld, -ret);
 			goto done;
 		}
 
@@ -73,10 +89,10 @@ static void fpga_image_do_load(struct work_struct *work)
 		offset += ret;
 	}
 
-	imgld->progress = FPGA_IMAGE_PROG_PROGRAMMING;
+	fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PROGRAMMING);
 	ret = imgld->ops->poll_complete(imgld);
 	if (ret)
-		imgld->err_code = ret;
+		fpga_image_set_error(imgld, ret);
 
 done:
 	if (imgld->ops->cleanup)
@@ -151,20 +167,42 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
 	return ret;
 }
 
+static int fpga_image_load_ioctl_status(struct fpga_image_load *imgld,
+					unsigned long arg)
+{
+	struct fpga_image_status status;
+
+	memset(&status, 0, sizeof(status));
+	status.progress = imgld->progress;
+	status.remaining_size = imgld->remaining_size;
+	status.err_progress = imgld->err_progress;
+	status.err_code = imgld->err_code;
+
+	if (copy_to_user((void __user *)arg, &status, sizeof(status)))
+		return -EFAULT;
+
+	return 0;
+}
+
 static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
 				  unsigned long arg)
 {
 	struct fpga_image_load *imgld = filp->private_data;
 	int ret = -ENOTTY;
 
+	mutex_lock(&imgld->lock);
+
 	switch (cmd) {
 	case FPGA_IMAGE_LOAD_WRITE:
-		mutex_lock(&imgld->lock);
 		ret = fpga_image_load_ioctl_write(imgld, arg);
-		mutex_unlock(&imgld->lock);
+		break;
+	case FPGA_IMAGE_LOAD_STATUS:
+		ret = fpga_image_load_ioctl_status(imgld, arg);
 		break;
 	}
 
+	mutex_unlock(&imgld->lock);
+
 	return ret;
 }
 
diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
index 77b3c91ce073..366111d090fb 100644
--- a/include/linux/fpga/fpga-image-load.h
+++ b/include/linux/fpga/fpga-image-load.h
@@ -49,6 +49,7 @@ struct fpga_image_load {
 	const u8 *data;			/* pointer to update data */
 	u32 remaining_size;		/* size remaining to transfer */
 	u32 progress;
+	u32 err_progress;		/* progress at time of error */
 	u32 err_code;			/* image load error code */
 	bool driver_unload;
 	struct eventfd_ctx *finished;
diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
index 8d2d3db92e87..1b91343961df 100644
--- a/include/uapi/linux/fpga-image-load.h
+++ b/include/uapi/linux/fpga-image-load.h
@@ -51,4 +51,22 @@ struct fpga_image_write {
 
 #define FPGA_IMAGE_LOAD_WRITE	_IOW(FPGA_IMAGE_LOAD_MAGIC, 0, struct fpga_image_write)
 
+/**
+ * FPGA_IMAGE_LOAD_STATUS - _IOR(FPGA_IMAGE_LOAD_MAGIC, 1,
+ *				 struct fpga_image_status)
+ *
+ * Request status information for an ongoing update.
+ *
+ * Return: 0 on success, -errno on failure.
+ */
+struct fpga_image_status {
+	/* Output */
+	__u32 remaining_size;	/* size remaining to transfer */
+	__u32 progress;		/* current progress of image load */
+	__u32 err_progress;	/* progress at time of error */
+	__u32 err_code;		/* error code */
+};
+
+#define FPGA_IMAGE_LOAD_STATUS	_IOR(FPGA_IMAGE_LOAD_MAGIC, 1, struct fpga_image_status)
+
 #endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */
-- 
2.25.1

[PATCH v17 5/5] fpga: image-load: enable cancel of image upload

[Russ Weight]

Extend the FPGA Image Load framework to include a cancel IOCTL that can be
used to request that an image upload be canceled. The IOCTL may return
EBUSY if it cannot be canceled by software or ENODEV if there is no update
in progress.

Signed-off-by: Russ Weight <russell.h.weight@intel.com>
---
v17:
 - Documentation cleanup for cancelling.
 - Removed the request_cancel flag and handling from the class driver
   including the fpga_image_prog_transition() function.
 - The cancel system call now directly calls the cancel() op of the
   lower-level driver. 
 - Changed the cancel() op to return void.
v16:
 - This was previously patch 6/6
 - Amend fpga_image_load_release() to request cancellation of an ongoing
   update when possible.
v15:
 - Compare to previous patch:
     [PATCH v14 6/6] fpga: sec-mgr: enable cancel of secure update
 - Changed file, symbol, and config names to reflect the new driver name
 - Cancel is now initiated by IOCT instead of sysfs
 - Removed signed-off/reviewed-by tags
v14:
 - Updated ABI documentation date and kernel version
v13:
  - No change
v12:
  - Updated Date and KernelVersion fields in ABI documentation
v11:
  - No change
v10:
  - Rebased to 5.12-rc2 next
  - Updated Date and KernelVersion in ABI documentation
v9:
  - Updated Date and KernelVersion in ABI documentation
v8:
  - No change
v7:
  - Changed Date in documentation file to December 2020
v6:
  - No change
v5:
  - No change
v4:
  - Changed from "Intel FPGA Security Manager" to FPGA Security Manager"
    and removed unnecessary references to "Intel".
  - Changed: iops -> sops, imgr -> smgr, IFPGA_ -> FPGA_, ifpga_ to fpga_
v3:
  - No change
v2:
  - Bumped documentation date and version
  - Minor code cleanup per review comments
---
 Documentation/fpga/fpga-image-load.rst | 13 +++++++++++--
 drivers/fpga/fpga-image-load.c         | 26 +++++++++++++++++++++++---
 include/linux/fpga/fpga-image-load.h   |  4 ++++
 include/uapi/linux/fpga-image-load.h   |  2 ++
 4 files changed, 40 insertions(+), 5 deletions(-)

diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
index f64f5ee473b8..3997b3bcd7ec 100644
--- a/Documentation/fpga/fpga-image-load.rst
+++ b/Documentation/fpga/fpga-image-load.rst
@@ -18,8 +18,9 @@ open
 
 An fpga_image_load device is opened exclusively to control an image upload.
 The device must remain open throughout the duration of the image upload.
-An attempt to close the device while an upload is in progress will block
-until the image upload is complete.
+An attempt to close the device while an upload is in progress will cause
+the upload to be cancelled. If unable to cancel the image upload, the close
+system call will block until the image upload is complete.
 
 ioctl
 -----
@@ -39,3 +40,11 @@ FPGA_IMAGE_LOAD_STATUS:
 Collect status for an on-going image upload. The status returned includes
 how much data remains to be transferred, the progress of the image upload,
 and error information in the case of a failure.
+
+FPGA_IMAGE_LOAD_CANCEL:
+
+Request that an on-going image upload be cancelled. This IOCTL will return
+ENODEV if there is no update in progress. Depending on the implementation
+of the lower-level driver, the cancellation may take affect immediately.
+In other cases, the cancellation request may be blocked until a critical
+operation such as a FLASH is safely completed.
diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
index 58373b9e8c02..239943d7f321 100644
--- a/drivers/fpga/fpga-image-load.c
+++ b/drivers/fpga/fpga-image-load.c
@@ -159,7 +159,6 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
 	imgld->err_code = 0;
 	imgld->progress = FPGA_IMAGE_PROG_STARTING;
 	queue_work(system_long_wq, &imgld->work);
-
 	return 0;
 
 exit_free:
@@ -184,11 +183,21 @@ static int fpga_image_load_ioctl_status(struct fpga_image_load *imgld,
 	return 0;
 }
 
+static int fpga_image_load_ioctl_cancel(struct fpga_image_load *imgld,
+					unsigned long arg)
+{
+	if (imgld->progress == FPGA_IMAGE_PROG_IDLE)
+		return -ENODEV;
+
+	imgld->ops->cancel(imgld);
+	return 0;
+}
+
 static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
 				  unsigned long arg)
 {
 	struct fpga_image_load *imgld = filp->private_data;
-	int ret = -ENOTTY;
+	int ret = 0;
 
 	mutex_lock(&imgld->lock);
 
@@ -199,6 +208,12 @@ static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
 	case FPGA_IMAGE_LOAD_STATUS:
 		ret = fpga_image_load_ioctl_status(imgld, arg);
 		break;
+	case FPGA_IMAGE_LOAD_CANCEL:
+		ret = fpga_image_load_ioctl_cancel(imgld, arg);
+		break;
+	default:
+		ret = -ENOTTY;
+		break;
 	}
 
 	mutex_unlock(&imgld->lock);
@@ -229,6 +244,8 @@ static int fpga_image_load_release(struct inode *inode, struct file *filp)
 		goto close_exit;
 	}
 
+	imgld->ops->cancel(imgld);
+
 	mutex_unlock(&imgld->lock);
 	flush_work(&imgld->work);
 
@@ -263,7 +280,8 @@ fpga_image_load_register(struct device *parent,
 	struct fpga_image_load *imgld;
 	int ret;
 
-	if (!ops || !ops->prepare || !ops->write || !ops->poll_complete) {
+	if (!ops || !ops->cancel || !ops->prepare ||
+	    !ops->write || !ops->poll_complete) {
 		dev_err(parent, "Attempt to register without all required ops\n");
 		return ERR_PTR(-ENOMEM);
 	}
@@ -342,6 +360,8 @@ void fpga_image_load_unregister(struct fpga_image_load *imgld)
 		goto unregister;
 	}
 
+	imgld->ops->cancel(imgld);
+
 	mutex_unlock(&imgld->lock);
 	flush_work(&imgld->work);
 
diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
index 366111d090fb..6baf45072bdb 100644
--- a/include/linux/fpga/fpga-image-load.h
+++ b/include/linux/fpga/fpga-image-load.h
@@ -26,6 +26,9 @@ struct fpga_image_load;
  *			    written.
  * @poll_complete:	    Required: Check for the completion of the
  *			    HW authentication/programming process.
+ * @cancel:		    Required: Request cancellation of update. This op
+ *			    is called from the context of a different kernel
+ *			    thread, so race conditions need to be considered.
  * @cleanup:		    Optional: Complements the prepare()
  *			    function and is called at the completion
  *			    of the update, whether success or failure,
@@ -36,6 +39,7 @@ struct fpga_image_load_ops {
 	s32 (*write)(struct fpga_image_load *imgld, const u8 *data,
 		     u32 offset, u32 size);
 	u32 (*poll_complete)(struct fpga_image_load *imgld);
+	void (*cancel)(struct fpga_image_load *imgld);
 	void (*cleanup)(struct fpga_image_load *imgld);
 };
 
diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
index 1b91343961df..5bf8a8a57757 100644
--- a/include/uapi/linux/fpga-image-load.h
+++ b/include/uapi/linux/fpga-image-load.h
@@ -69,4 +69,6 @@ struct fpga_image_status {
 
 #define FPGA_IMAGE_LOAD_STATUS	_IOR(FPGA_IMAGE_LOAD_MAGIC, 1, struct fpga_image_status)
 
+#define FPGA_IMAGE_LOAD_CANCEL	_IO(FPGA_IMAGE_LOAD_MAGIC, 2)
+
 #endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */
-- 
2.25.1

Re: [PATCH v17 2/5] fpga: image-load: enable image uploads

[Russ Weight]

On 9/29/21 4:00 PM, Russ Weight wrote:
> Extend the FPGA Image Load framework to include IOCTL support
> (FPGA_IMAGE_LOAD_WRITE) for initiating an upload of an image to a device.
> The IOCTL will return immediately, and the update will begin in the
> context of a kernel worker thread.
>
> Signed-off-by: Russ Weight <russell.h.weight@intel.com>
> ---
> v17:
>  - Minor documentation cleanup and references to cancellation are moved
>    to a later patch.
>  - We no longer send a "just in case" cancel on HW errors. The cleanup()
>    op can and should be used to manage the HW state on an error.
>  - Removed the FPGA_IMAGE_ERR_NONE macro in favor of returning zero. The
>    constant name made sense when we were using enum data types, but is no
>    longer necessary.
>  - Changed the syntax of the write_blk() function and renamed it to write().
>    It now returns a positive size or a negative error number.
>  - Changed the prepare() and write() ops to pass in elements of the
>    fpga_image_load data structure instead of having the op functions access
>    them directly via imgld.
>  - Removed the default block size. The lower-level driver must choose a
>    block size.
>  - Changed work queue from system_unbound_wq to system_long_wq.
>  - Removed calls to cond_resched(). The lower-level driver must manage
>    any scheduler issues.
>  - Use the cdev_set_parent() macro.
> v16:
>  - Shift from "Driver" terminology to "Framework" in comments and
>    documentation
>  - Rename lops to ops for structure member and local variables
>  - Change the write_blk() definition to pass in *blk_size (a pointer to
>    a default block size of WRITE_BLOCK_SIZE=0x4000) and max_size (the
>    the maximum block size to stay within the limit of the data buffer).
>    The write_blk() op may use the default *blk_size or modify it to a
>    more optimal number for the given device, subject to the max_size limit.
>  - All enum values for progress and errors are changed to macros, because
>    they are included in the uapi header. This is done to maintain consistency
>    amongst the DFL related IOCTL header files. All references to the enum
>    types are changed to u32.
>  - Bail out early in fpga_image_do_load() if imgld->driver_unload is true.
>  - Add a call to cond_resched() in the write_blk() loop to ensure that
>    we yield to higher priority tasks during long data transfers.
>  - Switch to the system_unbound_wq to enable calls to cond_resched().
>  - Switch from test_and_set_bit() to atomic_cmpxchg() to manage
>    imgld->opened.
>  - Change fpga_image_load_release() to block until the image upload is
>    complete.
>  - Remove the completion object, imgld->update_done, in favor of calling
>    flush_work(&imgld->work);
> v15:
>  - Compare to previous patch:
>      [PATCH v14 2/6] fpga: sec-mgr: enable secure updates
>  - Changed file, symbol, and config names to reflect the new driver name
>  - Removed update/filename sysfs file and added the FPGA_IMAGE_LOAD_WRITE
>    IOCTL for writing the image data to the FPGA card. The driver no longer
>    uses the request_firmware framework.
>  - Fixed some error return values in fpga_image_load_register()
>  - Removed signed-off/reviewed-by tags
> v14:
>  - Added MAINTAINERS reference for
>    Documentation/ABI/testing/sysfs-class-fpga-sec-mgr
>  - Updated ABI documentation date and kernel version
>  - Updated copyright to 2021
> v13:
>   - Change "if (count == 0 || " to "if (!count || "
>   - Improve error message: "Attempt to register without all required ops\n"
> v12:
>   - Updated Date and KernelVersion fields in ABI documentation
>   - Removed size parameter from write_blk() op - it is now up to
>     the lower-level driver to determine the appropriate size and
>     to update smgr->remaining_size accordingly.
> v11:
>   - Fixed a spelling error in a comment
>   - Initialize smgr->err_code and smgr->progress explicitly in
>     fpga_sec_mgr_create() instead of accepting the default 0 value.
> v10:
>   - Rebased to 5.12-rc2 next
>   - Updated Date and KernelVersion in ABI documentation
> v9:
>   - Updated Date and KernelVersion in ABI documentation
> v8:
>   - No change
> v7:
>   - Changed Date in documentation file to December 2020
>   - Changed filename_store() to use kmemdup_nul() instead of
>     kstrndup() and changed the count to not assume a line-return.
> v6:
>   - Changed "security update" to "secure update" in commit message
> v5:
>   - When checking the return values for functions of type enum
>     fpga_sec_err err_code, test for FPGA_SEC_ERR_NONE instead of 0
> v4:
>   - Changed from "Intel FPGA Security Manager" to FPGA Security Manager"
>     and removed unnecessary references to "Intel".
>   - Changed: iops -> sops, imgr -> smgr, IFPGA_ -> FPGA_, ifpga_ to fpga_
> v3:
>   - Removed unnecessary "goto done"
>   - Added a comment to explain imgr->driver_unload in
>     ifpga_sec_mgr_unregister()
> v2:
>   - Bumped documentation date and version
>   - Removed explicit value assignments in enums
>   - Other minor code cleanup per review comments
> ---
>  Documentation/fpga/fpga-image-load.rst |  25 ++-
>  MAINTAINERS                            |   1 +
>  drivers/fpga/fpga-image-load.c         | 217 ++++++++++++++++++++++++-
>  include/linux/fpga/fpga-image-load.h   |  27 +++
>  include/uapi/linux/fpga-image-load.h   |  53 ++++++
>  5 files changed, 321 insertions(+), 2 deletions(-)
>  create mode 100644 include/uapi/linux/fpga-image-load.h
>
> diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
> index dda9283ef1c7..f6b0e1624a05 100644
> --- a/Documentation/fpga/fpga-image-load.rst
> +++ b/Documentation/fpga/fpga-image-load.rst
> @@ -7,4 +7,27 @@ FPGA Image Load Framework
>  The FPGA Image Load framework provides a common API for user-space
>  tools to manage image uploads to FPGA devices. Device drivers that
>  instantiate the FPGA Image Load framework will interact with the
> -target device to transfer and authenticate the image data.
> +target device to transfer and authenticate the image data. Image uploads
> +are processed in the context of a kernel worker thread.
> +
> +User API
> +========
> +
> +open
> +----
> +
> +An fpga_image_load device is opened exclusively to control an image upload.
> +The device must remain open throughout the duration of the image upload.
> +An attempt to close the device while an upload is in progress will block
> +until the image upload is complete.
> +
> +ioctl
> +-----
> +
> +FPGA_IMAGE_LOAD_WRITE:
> +
> +Start an image upload with the provided image buffer. This IOCTL returns
> +immediately after starting a kernel worker thread to process the image
> +upload which could take as long as 40 minutes depending on the actual
> +device being updated. This is an exclusive operation; an attempt to start
> +concurrent image uploads for the same device will fail with EBUSY.
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a99622eebbff..e3b5c555ecbd 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -7415,6 +7415,7 @@ S:	Maintained
>  F:	Documentation/fpga/fpga-image-load.rst
>  F:	drivers/fpga/fpga-image-load.c
>  F:	include/linux/fpga/fpga-image-load.h
> +F:	include/uapi/linux/fpga-image-load.h
>  
>  FPU EMULATOR
>  M:	Bill Metzenthen <billm@melbpc.org.au>
> diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
> index 799d18444f7c..4c2571e3b26f 100644
> --- a/drivers/fpga/fpga-image-load.c
> +++ b/drivers/fpga/fpga-image-load.c
> @@ -5,18 +5,190 @@
>   * Copyright (C) 2019-2021 Intel Corporation, Inc.
>   */
>  
> +#include <linux/delay.h>
>  #include <linux/fpga/fpga-image-load.h>
> +#include <linux/fs.h>
> +#include <linux/kernel.h>
>  #include <linux/module.h>
>  #include <linux/slab.h>
> +#include <linux/uaccess.h>
>  #include <linux/vmalloc.h>
>  
>  #define IMAGE_LOAD_XA_LIMIT	XA_LIMIT(0, INT_MAX)
>  static DEFINE_XARRAY_ALLOC(fpga_image_load_xa);
>  
>  static struct class *fpga_image_load_class;
> +static dev_t fpga_image_devt;
>  
>  #define to_image_load(d) container_of(d, struct fpga_image_load, dev)
>  
> +static void fpga_image_prog_complete(struct fpga_image_load *imgld)
> +{
> +	mutex_lock(&imgld->lock);
> +	imgld->progress = FPGA_IMAGE_PROG_IDLE;
> +	mutex_unlock(&imgld->lock);
> +}
> +
> +static void fpga_image_do_load(struct work_struct *work)
> +{
> +	struct fpga_image_load *imgld;
> +	s32 ret, offset = 0;
> +
> +	imgld = container_of(work, struct fpga_image_load, work);
> +
> +	if (imgld->driver_unload) {
> +		imgld->err_code = FPGA_IMAGE_ERR_CANCELED;
> +		goto idle_exit;
> +	}
> +
> +	get_device(&imgld->dev);
> +	if (!try_module_get(imgld->dev.parent->driver->owner)) {
> +		imgld->err_code = FPGA_IMAGE_ERR_BUSY;
> +		goto putdev_exit;
> +	}
> +
> +	imgld->progress = FPGA_IMAGE_PROG_PREPARING;
> +	ret = imgld->ops->prepare(imgld, imgld->data, imgld->remaining_size);
> +	if (ret) {
> +		imgld->err_code = ret;
> +		goto modput_exit;
> +	}
> +
> +	imgld->progress = FPGA_IMAGE_PROG_WRITING;
> +	while (imgld->remaining_size) {
> +		ret = imgld->ops->write(imgld, imgld->data, offset,
> +					imgld->remaining_size);
> +		if (ret <= 0) {
> +			if (!ret) {
> +				dev_warn(&imgld->dev,
> +					 "write-op wrote zero data\n");
> +				ret = -FPGA_IMAGE_ERR_RW_ERROR;
> +			}
> +			imgld->err_code = -ret;
> +			goto done;
> +		}
> +
> +		imgld->remaining_size -= ret;
> +		offset += ret;
> +	}
> +
> +	imgld->progress = FPGA_IMAGE_PROG_PROGRAMMING;
> +	ret = imgld->ops->poll_complete(imgld);
> +	if (ret)
> +		imgld->err_code = ret;
> +
> +done:
> +	if (imgld->ops->cleanup)
> +		imgld->ops->cleanup(imgld);
> +
> +modput_exit:
> +	module_put(imgld->dev.parent->driver->owner);
> +
> +putdev_exit:
> +	put_device(&imgld->dev);
> +
> +idle_exit:
> +	/*
> +	 * Note: imgld->remaining_size is left unmodified here to provide
> +	 * additional information on errors. It will be reinitialized when
> +	 * the next image load begins.
> +	 */
> +	vfree(imgld->data);
> +	imgld->data = NULL;
> +	fpga_image_prog_complete(imgld);
> +}
> +
> +static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
> +				       unsigned long arg)
> +{
> +	struct fpga_image_write wb;
> +	unsigned long minsz;
> +	u8 *buf;
> +
> +	if (imgld->driver_unload || imgld->progress != FPGA_IMAGE_PROG_IDLE)
> +		return -EBUSY;
> +
> +	minsz = offsetofend(struct fpga_image_write, buf);
> +	if (copy_from_user(&wb, (void __user *)arg, minsz))
> +		return -EFAULT;
> +
> +	if (wb.flags)
> +		return -EINVAL;
FYI: Static analysis flagged that wb.size is not being validated
before being used by vzalloc. It is a 32-bit unsigned value, so
size is probably not a concern. If there are size or alignment
constraints, they belong to the lower-level driver and should
be checked there. I propose to add a null check here:

+       if (!wb.size)
+               return -EINVAL;
+

I'll await more comments before I re-roll the patch set.

- Russ

> +
> +	buf = vzalloc(wb.size);
> +	if (!buf)
> +		return -ENOMEM;
> +
> +	if (copy_from_user(buf, u64_to_user_ptr(wb.buf), wb.size)) {
> +		vfree(buf);
> +		return -EFAULT;
> +	}
> +
> +	imgld->data = buf;
> +	imgld->remaining_size = wb.size;
> +	imgld->err_code = 0;
> +	imgld->progress = FPGA_IMAGE_PROG_STARTING;
> +	queue_work(system_long_wq, &imgld->work);
> +
> +	return 0;
> +}
> +
> +static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
> +				  unsigned long arg)
> +{
> +	struct fpga_image_load *imgld = filp->private_data;
> +	int ret = -ENOTTY;
> +
> +	switch (cmd) {
> +	case FPGA_IMAGE_LOAD_WRITE:
> +		mutex_lock(&imgld->lock);
> +		ret = fpga_image_load_ioctl_write(imgld, arg);
> +		mutex_unlock(&imgld->lock);
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static int fpga_image_load_open(struct inode *inode, struct file *filp)
> +{
> +	struct fpga_image_load *imgld = container_of(inode->i_cdev,
> +						     struct fpga_image_load, cdev);
> +
> +	if (atomic_cmpxchg(&imgld->opened, 0, 1))
> +		return -EBUSY;
> +
> +	filp->private_data = imgld;
> +
> +	return 0;
> +}
> +
> +static int fpga_image_load_release(struct inode *inode, struct file *filp)
> +{
> +	struct fpga_image_load *imgld = filp->private_data;
> +
> +	mutex_lock(&imgld->lock);
> +	if (imgld->progress == FPGA_IMAGE_PROG_IDLE) {
> +		mutex_unlock(&imgld->lock);
> +		goto close_exit;
> +	}
> +
> +	mutex_unlock(&imgld->lock);
> +	flush_work(&imgld->work);
> +
> +close_exit:
> +	atomic_set(&imgld->opened, 0);
> +
> +	return 0;
> +}
> +
> +static const struct file_operations fpga_image_load_fops = {
> +	.owner = THIS_MODULE,
> +	.open = fpga_image_load_open,
> +	.release = fpga_image_load_release,
> +	.unlocked_ioctl = fpga_image_load_ioctl,
> +};
> +
>  /**
>   * fpga_image_load_register - create and register an FPGA Image Load Device
>   *
> @@ -35,6 +207,11 @@ fpga_image_load_register(struct device *parent,
>  	struct fpga_image_load *imgld;
>  	int ret;
>  
> +	if (!ops || !ops->prepare || !ops->write || !ops->poll_complete) {
> +		dev_err(parent, "Attempt to register without all required ops\n");
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
>  	imgld = kzalloc(sizeof(*imgld), GFP_KERNEL);
>  	if (!imgld)
>  		return ERR_PTR(-ENOMEM);
> @@ -48,9 +225,13 @@ fpga_image_load_register(struct device *parent,
>  
>  	imgld->priv = priv;
>  	imgld->ops = ops;
> +	imgld->err_code = 0;
> +	imgld->progress = FPGA_IMAGE_PROG_IDLE;
> +	INIT_WORK(&imgld->work, fpga_image_do_load);
>  
>  	imgld->dev.class = fpga_image_load_class;
>  	imgld->dev.parent = parent;
> +	imgld->dev.devt = MKDEV(MAJOR(fpga_image_devt), imgld->dev.id);
>  
>  	ret = dev_set_name(&imgld->dev, "fpga_image_load%d", imgld->dev.id);
>  	if (ret) {
> @@ -65,6 +246,16 @@ fpga_image_load_register(struct device *parent,
>  		return ERR_PTR(ret);
>  	}
>  
> +	cdev_init(&imgld->cdev, &fpga_image_load_fops);
> +	imgld->cdev.owner = parent->driver->owner;
> +	cdev_set_parent(&imgld->cdev, &imgld->dev.kobj);
> +
> +	ret = cdev_add(&imgld->cdev, imgld->dev.devt, 1);
> +	if (ret) {
> +		put_device(&imgld->dev);
> +		return ERR_PTR(ret);
> +	}
> +
>  	return imgld;
>  
>  error_device:
> @@ -83,10 +274,23 @@ EXPORT_SYMBOL_GPL(fpga_image_load_register);
>   * @imgld: pointer to struct fpga_image_load
>   *
>   * This function is intended for use in the parent driver's remove()
> - * function.
> + * function. The driver_unload flag prevents new updates from starting
> + * once the unregister process has begun.
>   */
>  void fpga_image_load_unregister(struct fpga_image_load *imgld)
>  {
> +	mutex_lock(&imgld->lock);
> +	imgld->driver_unload = true;
> +	if (imgld->progress == FPGA_IMAGE_PROG_IDLE) {
> +		mutex_unlock(&imgld->lock);
> +		goto unregister;
> +	}
> +
> +	mutex_unlock(&imgld->lock);
> +	flush_work(&imgld->work);
> +
> +unregister:
> +	cdev_del(&imgld->cdev);
>  	device_unregister(&imgld->dev);
>  }
>  EXPORT_SYMBOL_GPL(fpga_image_load_unregister);
> @@ -101,19 +305,30 @@ static void fpga_image_load_dev_release(struct device *dev)
>  
>  static int __init fpga_image_load_class_init(void)
>  {
> +	int ret;
>  	pr_info("FPGA Image Load Framework\n");
>  
>  	fpga_image_load_class = class_create(THIS_MODULE, "fpga_image_load");
>  	if (IS_ERR(fpga_image_load_class))
>  		return PTR_ERR(fpga_image_load_class);
>  
> +	ret = alloc_chrdev_region(&fpga_image_devt, 0, MINORMASK,
> +				  "fpga_image_load");
> +	if (ret)
> +		goto exit_destroy_class;
> +
>  	fpga_image_load_class->dev_release = fpga_image_load_dev_release;
>  
>  	return 0;
> +
> +exit_destroy_class:
> +	class_destroy(fpga_image_load_class);
> +	return ret;
>  }
>  
>  static void __exit fpga_image_load_class_exit(void)
>  {
> +	unregister_chrdev_region(fpga_image_devt, MINORMASK);
>  	class_destroy(fpga_image_load_class);
>  	WARN_ON(!xa_empty(&fpga_image_load_xa));
>  }
> diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
> index 8b051c82ef5f..7b445f489644 100644
> --- a/include/linux/fpga/fpga-image-load.h
> +++ b/include/linux/fpga/fpga-image-load.h
> @@ -7,22 +7,49 @@
>  #ifndef _LINUX_FPGA_IMAGE_LOAD_H
>  #define _LINUX_FPGA_IMAGE_LOAD_H
>  
> +#include <linux/cdev.h>
>  #include <linux/device.h>
>  #include <linux/mutex.h>
>  #include <linux/types.h>
> +#include <uapi/linux/fpga-image-load.h>
>  
>  struct fpga_image_load;
>  
>  /**
>   * struct fpga_image_load_ops - device specific operations
> + * @prepare:		    Required: Prepare secure update
> + * @write:		    Required: The write() op receives the remaining
> + *			    size to be written and must return the actual
> + *			    size written or a negative error code. The write()
> + *			    op will be called repeatedly until all data is
> + *			    written.
> + * @poll_complete:	    Required: Check for the completion of the
> + *			    HW authentication/programming process.
> + * @cleanup:		    Optional: Complements the prepare()
> + *			    function and is called at the completion
> + *			    of the update, whether success or failure,
> + *			    if the prepare function succeeded.
>   */
>  struct fpga_image_load_ops {
> +	u32 (*prepare)(struct fpga_image_load *imgld, const u8 *data, u32 size);
> +	s32 (*write)(struct fpga_image_load *imgld, const u8 *data,
> +		     u32 offset, u32 size);
> +	u32 (*poll_complete)(struct fpga_image_load *imgld);
> +	void (*cleanup)(struct fpga_image_load *imgld);
>  };
>  
>  struct fpga_image_load {
>  	struct device dev;
> +	struct cdev cdev;
>  	const struct fpga_image_load_ops *ops;
>  	struct mutex lock;		/* protect data structure contents */
> +	atomic_t opened;
> +	struct work_struct work;
> +	const u8 *data;			/* pointer to update data */
> +	u32 remaining_size;		/* size remaining to transfer */
> +	u32 progress;
> +	u32 err_code;			/* image load error code */
> +	bool driver_unload;
>  	void *priv;
>  };
>  
> diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
> new file mode 100644
> index 000000000000..20eae3bb10d8
> --- /dev/null
> +++ b/include/uapi/linux/fpga-image-load.h
> @@ -0,0 +1,53 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * Header File for FPGA Image Load User API
> + *
> + * Copyright (C) 2019-2021 Intel Corporation, Inc.
> + *
> + */
> +
> +#ifndef _UAPI_LINUX_FPGA_IMAGE_LOAD_H
> +#define _UAPI_LINUX_FPGA_IMAGE_LOAD_H
> +
> +#include <linux/types.h>
> +#include <linux/ioctl.h>
> +
> +#define FPGA_IMAGE_LOAD_MAGIC 0xB9
> +
> +/* Image load progress codes */
> +#define FPGA_IMAGE_PROG_IDLE		0
> +#define FPGA_IMAGE_PROG_STARTING	1
> +#define FPGA_IMAGE_PROG_PREPARING	2
> +#define FPGA_IMAGE_PROG_WRITING		3
> +#define FPGA_IMAGE_PROG_PROGRAMMING	4
> +#define FPGA_IMAGE_PROG_MAX		5
> +
> +/* Image error progress codes */
> +#define FPGA_IMAGE_ERR_HW_ERROR		1
> +#define FPGA_IMAGE_ERR_TIMEOUT		2
> +#define FPGA_IMAGE_ERR_CANCELED		3
> +#define FPGA_IMAGE_ERR_BUSY		4
> +#define FPGA_IMAGE_ERR_INVALID_SIZE	5
> +#define FPGA_IMAGE_ERR_RW_ERROR		6
> +#define FPGA_IMAGE_ERR_WEAROUT		7
> +#define FPGA_IMAGE_ERR_MAX		8
> +
> +/**
> + * FPGA_IMAGE_LOAD_WRITE - _IOW(FPGA_IMAGE_LOAD_MAGIC, 0,
> + *				struct fpga_image_write)
> + *
> + * Upload a data buffer to the target device. The user must provide the
> + * data buffer and size.
> + *
> + * Return: 0 on success, -errno on failure.
> + */
> +struct fpga_image_write {
> +	/* Input */
> +	__u32 flags;		/* Zero for now */
> +	__u32 size;		/* Data size (in bytes) to be written */
> +	__u64 buf;		/* User space address of source data */
> +};
> +
> +#define FPGA_IMAGE_LOAD_WRITE	_IOW(FPGA_IMAGE_LOAD_MAGIC, 0, struct fpga_image_write)
> +
> +#endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> The FPGA Image Load framework provides an API to upload image
> files to an FPGA device. Image files are self-describing. They could
> contain FPGA images, BMC images, Root Entry Hashes, or other device
> specific files. It is up to the lower-level device driver and the
> target device to authenticate and disposition the file data.

I've reconsider the FPGA persistent image update again, and think we
may include it in FPGA manager framework.

Sorry I raised this topic again when it is already at patch v17, but now
I need to consider more seriously than before.

We have consensus the FPGA persistent image update is just like a normal
firmware update which finally writes the nvmem like flash or eeprom,
while the current FPGA manager deals with the active FPGA region update
and re-activation. Could we just expand the FPGA manager and let it handle
the nvmem update as well? Many FPGA cards have nvmem and downloaders
supports updating both FPGA region and nvmem.

According to the patchset, the basic workflow of the 2 update types are
quite similar, get the data, prepare for the HW, write and complete.
They are already implemented in FPGA manager. We've discussed some
differences like threading or canceling the update, which are
not provided by FPGA manager but they may also nice to have for FPGA
region update. An FPGA region update may also last for a long time??
So I think having 2 sets of similar frameworks in FPGA is unnecessary.

My quick mind is that we add some flags in struct fpga_mgr & struct
fpga_image_info to indicate the HW capability (support FPGA region
update or nvmem update or both) of the download engine and the provided
image type. Then the low-level driver knows how to download if it
supports both image types.

An char device could be added for each fpga manager dev, providing the
user APIs for nvmem update. We may not use the char dev for FPGA region
update cause it changes the system HW devices and needs device hotplug
in FPGA region. We'd better leave it to FPGA region class, this is
another topic.

More discussion is appreciated.

Thanks,
Yilun

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Tom Rix]

On 10/9/21 1:08 AM, Xu Yilun wrote:
> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>> The FPGA Image Load framework provides an API to upload image
>> files to an FPGA device. Image files are self-describing. They could
>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>> specific files. It is up to the lower-level device driver and the
>> target device to authenticate and disposition the file data.
> I've reconsider the FPGA persistent image update again, and think we
> may include it in FPGA manager framework.
>
> Sorry I raised this topic again when it is already at patch v17, but now
> I need to consider more seriously than before.
>
> We have consensus the FPGA persistent image update is just like a normal
> firmware update which finally writes the nvmem like flash or eeprom,
> while the current FPGA manager deals with the active FPGA region update
> and re-activation. Could we just expand the FPGA manager and let it handle
> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> supports updating both FPGA region and nvmem.
>
> According to the patchset, the basic workflow of the 2 update types are
> quite similar, get the data, prepare for the HW, write and complete.
> They are already implemented in FPGA manager. We've discussed some
> differences like threading or canceling the update, which are
> not provided by FPGA manager but they may also nice to have for FPGA
> region update. An FPGA region update may also last for a long time??
> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>
> My quick mind is that we add some flags in struct fpga_mgr & struct
> fpga_image_info to indicate the HW capability (support FPGA region
> update or nvmem update or both) of the download engine and the provided
> image type. Then the low-level driver knows how to download if it
> supports both image types.
>
> An char device could be added for each fpga manager dev, providing the
> user APIs for nvmem update. We may not use the char dev for FPGA region
> update cause it changes the system HW devices and needs device hotplug
> in FPGA region. We'd better leave it to FPGA region class, this is
> another topic.
>
> More discussion is appreciated.

I also think fpga_mgr could be extended.

In this patchset,

https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/

A second, similar set of write ops was added to fpga_manger_ops,

new bit/flag was added to fpga_image_info

The intent was for dfl to add their specific ops to cover what is done 
in this patchset.

Any other driver would do similar.

Is this close to what you are thinking ?

Tom

>
> Thanks,
> Yilun
>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> 
> On 10/9/21 1:08 AM, Xu Yilun wrote:
> > On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> > > The FPGA Image Load framework provides an API to upload image
> > > files to an FPGA device. Image files are self-describing. They could
> > > contain FPGA images, BMC images, Root Entry Hashes, or other device
> > > specific files. It is up to the lower-level device driver and the
> > > target device to authenticate and disposition the file data.
> > I've reconsider the FPGA persistent image update again, and think we
> > may include it in FPGA manager framework.
> > 
> > Sorry I raised this topic again when it is already at patch v17, but now
> > I need to consider more seriously than before.
> > 
> > We have consensus the FPGA persistent image update is just like a normal
> > firmware update which finally writes the nvmem like flash or eeprom,
> > while the current FPGA manager deals with the active FPGA region update
> > and re-activation. Could we just expand the FPGA manager and let it handle
> > the nvmem update as well? Many FPGA cards have nvmem and downloaders
> > supports updating both FPGA region and nvmem.
> > 
> > According to the patchset, the basic workflow of the 2 update types are
> > quite similar, get the data, prepare for the HW, write and complete.
> > They are already implemented in FPGA manager. We've discussed some
> > differences like threading or canceling the update, which are
> > not provided by FPGA manager but they may also nice to have for FPGA
> > region update. An FPGA region update may also last for a long time??
> > So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> > 
> > My quick mind is that we add some flags in struct fpga_mgr & struct
> > fpga_image_info to indicate the HW capability (support FPGA region
> > update or nvmem update or both) of the download engine and the provided
> > image type. Then the low-level driver knows how to download if it
> > supports both image types.
> > 
> > An char device could be added for each fpga manager dev, providing the
> > user APIs for nvmem update. We may not use the char dev for FPGA region
> > update cause it changes the system HW devices and needs device hotplug
> > in FPGA region. We'd better leave it to FPGA region class, this is
> > another topic.
> > 
> > More discussion is appreciated.
> 
> I also think fpga_mgr could be extended.
> 
> In this patchset,
> 
> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> 
> A second, similar set of write ops was added to fpga_manger_ops,
> 
> new bit/flag was added to fpga_image_info
> 
> The intent was for dfl to add their specific ops to cover what is done in
> this patchset.

I think we don't have to add 2 ops for reconfig & reimage in framework,
the 2 processes are almost the same.

Just add the _REIMAGE (or something else, NVMEM?) flag for
fpga_image_info, and low level drivers handle it as they do for other
flags.

How do you think?

Thanks,
Yilun

> 
> Any other driver would do similar.
> 
> Is this close to what you are thinking ?
> 
> Tom
> 
> > 
> > Thanks,
> > Yilun
> > 

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Tom Rix]

On 10/10/21 6:41 PM, Xu Yilun wrote:
> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>> The FPGA Image Load framework provides an API to upload image
>>>> files to an FPGA device. Image files are self-describing. They could
>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>> specific files. It is up to the lower-level device driver and the
>>>> target device to authenticate and disposition the file data.
>>> I've reconsider the FPGA persistent image update again, and think we
>>> may include it in FPGA manager framework.
>>>
>>> Sorry I raised this topic again when it is already at patch v17, but now
>>> I need to consider more seriously than before.
>>>
>>> We have consensus the FPGA persistent image update is just like a normal
>>> firmware update which finally writes the nvmem like flash or eeprom,
>>> while the current FPGA manager deals with the active FPGA region update
>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>> supports updating both FPGA region and nvmem.
>>>
>>> According to the patchset, the basic workflow of the 2 update types are
>>> quite similar, get the data, prepare for the HW, write and complete.
>>> They are already implemented in FPGA manager. We've discussed some
>>> differences like threading or canceling the update, which are
>>> not provided by FPGA manager but they may also nice to have for FPGA
>>> region update. An FPGA region update may also last for a long time??
>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>
>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>> fpga_image_info to indicate the HW capability (support FPGA region
>>> update or nvmem update or both) of the download engine and the provided
>>> image type. Then the low-level driver knows how to download if it
>>> supports both image types.
>>>
>>> An char device could be added for each fpga manager dev, providing the
>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>> update cause it changes the system HW devices and needs device hotplug
>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>> another topic.
>>>
>>> More discussion is appreciated.
>> I also think fpga_mgr could be extended.
>>
>> In this patchset,
>>
>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>
>> A second, similar set of write ops was added to fpga_manger_ops,
>>
>> new bit/flag was added to fpga_image_info
>>
>> The intent was for dfl to add their specific ops to cover what is done in
>> this patchset.
> I think we don't have to add 2 ops for reconfig & reimage in framework,
> the 2 processes are almost the same.
>
> Just add the _REIMAGE (or something else, NVMEM?) flag for
> fpga_image_info, and low level drivers handle it as they do for other
> flags.
>
> How do you think?

A single set is fine.

A difficult part of is the length of  time to do the write. The existing 
write should be improved to use a worker thread.

Tom

>
> Thanks,
> Yilun
>
>> Any other driver would do similar.
>>
>> Is this close to what you are thinking ?
>>
>> Tom
>>
>>> Thanks,
>>> Yilun
>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/11/21 5:35 AM, Tom Rix wrote:
>
> On 10/10/21 6:41 PM, Xu Yilun wrote:
>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>> The FPGA Image Load framework provides an API to upload image
>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>> specific files. It is up to the lower-level device driver and the
>>>>> target device to authenticate and disposition the file data.
>>>> I've reconsider the FPGA persistent image update again, and think we
>>>> may include it in FPGA manager framework.
>>>>
>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>> I need to consider more seriously than before.
>>>>
>>>> We have consensus the FPGA persistent image update is just like a normal
>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>> while the current FPGA manager deals with the active FPGA region update
>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>> supports updating both FPGA region and nvmem.
The fpga-image-load driver is actually just a data transfer. The class
driver has no knowledge about what the data is or where/if the data will
be stored.

This functionality could, of course, be merged into the fpga-mgr. I did
a proof of concept of this a while back and we discussed the pros and cons.
See this email for a recap:

https://marc.info/?l=linux-fpga&m=161998085507374&w=2

Things have changed some with the evolution of the driver. The IOCTL
approach probably fits better than the sysfs implementation. At the time
it seemed that a merge would add unnecessary complexity without adding value.

>>>>
>>>> According to the patchset, the basic workflow of the 2 update types are
>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>> They are already implemented in FPGA manager. We've discussed some
>>>> differences like threading or canceling the update, which are
>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>> region update. An FPGA region update may also last for a long time??
>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>
>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>> update or nvmem update or both) of the download engine and the provided
>>>> image type. Then the low-level driver knows how to download if it
>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>> update cause it changes the system HW devices and needs device hotplug
>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>> another topic.
I'll give this some more thought and see if I can come up with some RFC
patches.

- Russ
>>>>
>>>> More discussion is appreciated.
>>> I also think fpga_mgr could be extended.
>>>
>>> In this patchset,
>>>
>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>
>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>
>>> new bit/flag was added to fpga_image_info
>>>
>>> The intent was for dfl to add their specific ops to cover what is done in
>>> this patchset.
>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>> the 2 processes are almost the same.
>>
>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>> fpga_image_info, and low level drivers handle it as they do for other
>> flags.
>>
>> How do you think?
>
> A single set is fine.
>
> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>
> Tom
>
>>
>> Thanks,
>> Yilun
>>
>>> Any other driver would do similar.
>>>
>>> Is this close to what you are thinking ?
>>>
>>> Tom
>>>
>>>> Thanks,
>>>> Yilun
>>>>
>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> 
> 
> On 10/11/21 5:35 AM, Tom Rix wrote:
> >
> > On 10/10/21 6:41 PM, Xu Yilun wrote:
> >> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>> The FPGA Image Load framework provides an API to upload image
> >>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>> specific files. It is up to the lower-level device driver and the
> >>>>> target device to authenticate and disposition the file data.
> >>>> I've reconsider the FPGA persistent image update again, and think we
> >>>> may include it in FPGA manager framework.
> >>>>
> >>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>> I need to consider more seriously than before.
> >>>>
> >>>> We have consensus the FPGA persistent image update is just like a normal
> >>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>> while the current FPGA manager deals with the active FPGA region update
> >>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>> supports updating both FPGA region and nvmem.
> The fpga-image-load driver is actually just a data transfer. The class

IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
acting as the FPGA region admin responsible for gating, transfering and
re-enumerating.

So my opinion is to add a new data transfer type and keep a unified process.

> driver has no knowledge about what the data is or where/if the data will
> be stored.

The fpga-image-load driver knows the data will be stored in nvmem, while
the fpga-mgr knows the data will be stored in FPGA cells. They may need
to know the exact physical position to store, may not, depends on what the
HW engines are.

> 
> This functionality could, of course, be merged into the fpga-mgr. I did
> a proof of concept of this a while back and we discussed the pros and cons.
> See this email for a recap:
> 
> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> 
> Things have changed some with the evolution of the driver. The IOCTL
> approach probably fits better than the sysfs implementation. At the time
> it seemed that a merge would add unnecessary complexity without adding value.

I think at least developers don't have to go through 2 sets of software
stacks which are of the same concept. And adding some new features like
optionally threading or canceling data transfer are also good to FPGA
region update. And the nvmem update could also be benifit from exsiting
implementations like scatter-gather buffers, in-kernel firmware loading.

I try to explain myself according to each of your concern from that mail
thread:

Purpose of the 2 updates
========================

  As I said before, I think they are both data transfer devices. FPGA
region update gets extra support from fpga-region & fpga-bridge, and
FPGA nvmem update could be a standalone fpga-mgr.

Extra APIs that are unique to nvmem update
==========================================

  cdev APIs for nvmem update:
    Yes we need to expand the functionality so we need them.

  available_images, image_load APIs for loading nvmem content to FPGA
  region:
    These are features in later patchsets which are not submitted, but we
    could talk about it in advance. I think this is actually a FPGA region
    update from nvmem, it also requires gating, data loading (no SW transfer)
    and re-enumeration, or a single command to image_load HW may result system
    disordered. The FPGA framework now only supports update from in-kernel
    user data, maybe we add support for update from nvmem later.

  fpga-mgr state extend:
    I think it could be extended, The current states are not perfect,
    adding something like IDLE or READY is just fine.

  fpga-mgr status extend:
    Add general error definitions as needed. If there is some status
    that is quite vendor specific, expose it in low-level driver.

  remaining-size:
    Nice to have for all.

Threading the update
====================

  Also a good option for FPGA region update, maybe we also have a slow FPGA
  reprogrammer?

Cancelling the update
====================

  Also a good option for FPGA region update if HW engine supports.

Thanks,
Yilun

> 
> >>>>
> >>>> According to the patchset, the basic workflow of the 2 update types are
> >>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>> They are already implemented in FPGA manager. We've discussed some
> >>>> differences like threading or canceling the update, which are
> >>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>> region update. An FPGA region update may also last for a long time??
> >>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>
> >>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>> update or nvmem update or both) of the download engine and the provided
> >>>> image type. Then the low-level driver knows how to download if it
> >>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>> update cause it changes the system HW devices and needs device hotplug
> >>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>> another topic.
> I'll give this some more thought and see if I can come up with some RFC
> patches.
> 
> - Russ
> >>>>
> >>>> More discussion is appreciated.
> >>> I also think fpga_mgr could be extended.
> >>>
> >>> In this patchset,
> >>>
> >>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>
> >>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>
> >>> new bit/flag was added to fpga_image_info
> >>>
> >>> The intent was for dfl to add their specific ops to cover what is done in
> >>> this patchset.
> >> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >> the 2 processes are almost the same.
> >>
> >> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >> fpga_image_info, and low level drivers handle it as they do for other
> >> flags.
> >>
> >> How do you think?
> >
> > A single set is fine.
> >
> > A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >
> > Tom
> >
> >>
> >> Thanks,
> >> Yilun
> >>
> >>> Any other driver would do similar.
> >>>
> >>> Is this close to what you are thinking ?
> >>>
> >>> Tom
> >>>
> >>>> Thanks,
> >>>> Yilun
> >>>>
> >

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Mon, Oct 11, 2021 at 05:35:03AM -0700, Tom Rix wrote:
> 
> On 10/10/21 6:41 PM, Xu Yilun wrote:
> > On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> > > On 10/9/21 1:08 AM, Xu Yilun wrote:
> > > > On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> > > > > The FPGA Image Load framework provides an API to upload image
> > > > > files to an FPGA device. Image files are self-describing. They could
> > > > > contain FPGA images, BMC images, Root Entry Hashes, or other device
> > > > > specific files. It is up to the lower-level device driver and the
> > > > > target device to authenticate and disposition the file data.
> > > > I've reconsider the FPGA persistent image update again, and think we
> > > > may include it in FPGA manager framework.
> > > > 
> > > > Sorry I raised this topic again when it is already at patch v17, but now
> > > > I need to consider more seriously than before.
> > > > 
> > > > We have consensus the FPGA persistent image update is just like a normal
> > > > firmware update which finally writes the nvmem like flash or eeprom,
> > > > while the current FPGA manager deals with the active FPGA region update
> > > > and re-activation. Could we just expand the FPGA manager and let it handle
> > > > the nvmem update as well? Many FPGA cards have nvmem and downloaders
> > > > supports updating both FPGA region and nvmem.
> > > > 
> > > > According to the patchset, the basic workflow of the 2 update types are
> > > > quite similar, get the data, prepare for the HW, write and complete.
> > > > They are already implemented in FPGA manager. We've discussed some
> > > > differences like threading or canceling the update, which are
> > > > not provided by FPGA manager but they may also nice to have for FPGA
> > > > region update. An FPGA region update may also last for a long time??
> > > > So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> > > > 
> > > > My quick mind is that we add some flags in struct fpga_mgr & struct
> > > > fpga_image_info to indicate the HW capability (support FPGA region
> > > > update or nvmem update or both) of the download engine and the provided
> > > > image type. Then the low-level driver knows how to download if it
> > > > supports both image types.
> > > > 
> > > > An char device could be added for each fpga manager dev, providing the
> > > > user APIs for nvmem update. We may not use the char dev for FPGA region
> > > > update cause it changes the system HW devices and needs device hotplug
> > > > in FPGA region. We'd better leave it to FPGA region class, this is
> > > > another topic.
> > > > 
> > > > More discussion is appreciated.
> > > I also think fpga_mgr could be extended.
> > > 
> > > In this patchset,
> > > 
> > > https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> > > 
> > > A second, similar set of write ops was added to fpga_manger_ops,
> > > 
> > > new bit/flag was added to fpga_image_info
> > > 
> > > The intent was for dfl to add their specific ops to cover what is done in
> > > this patchset.
> > I think we don't have to add 2 ops for reconfig & reimage in framework,
> > the 2 processes are almost the same.
> > 
> > Just add the _REIMAGE (or something else, NVMEM?) flag for
> > fpga_image_info, and low level drivers handle it as they do for other
> > flags.
> > 
> > How do you think?
> 
> A single set is fine.
> 
> A difficult part of is the length of  time to do the write. The existing
> write should be improved to use a worker thread.

Yes, we could have a flag and optionally threading the writing.

Thanks,
Yilun

> 
> Tom
> 
> > 
> > Thanks,
> > Yilun
> > 
> > > Any other driver would do similar.
> > > 
> > > Is this close to what you are thinking ?
> > > 
> > > Tom
> > > 
> > > > Thanks,
> > > > Yilun
> > > > 

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Tue, Oct 12, 2021 at 03:47:52PM +0800, Xu Yilun wrote:
> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> > 
> > 
> > On 10/11/21 5:35 AM, Tom Rix wrote:
> > >
> > > On 10/10/21 6:41 PM, Xu Yilun wrote:
> > >> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> > >>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> > >>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> > >>>>> The FPGA Image Load framework provides an API to upload image
> > >>>>> files to an FPGA device. Image files are self-describing. They could
> > >>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> > >>>>> specific files. It is up to the lower-level device driver and the
> > >>>>> target device to authenticate and disposition the file data.
> > >>>> I've reconsider the FPGA persistent image update again, and think we
> > >>>> may include it in FPGA manager framework.
> > >>>>
> > >>>> Sorry I raised this topic again when it is already at patch v17, but now
> > >>>> I need to consider more seriously than before.
> > >>>>
> > >>>> We have consensus the FPGA persistent image update is just like a normal
> > >>>> firmware update which finally writes the nvmem like flash or eeprom,
> > >>>> while the current FPGA manager deals with the active FPGA region update
> > >>>> and re-activation. Could we just expand the FPGA manager and let it handle
> > >>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> > >>>> supports updating both FPGA region and nvmem.
> > The fpga-image-load driver is actually just a data transfer. The class
> 
> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> acting as the FPGA region admin responsible for gating, transfering and
> re-enumerating.
> 
> So my opinion is to add a new data transfer type and keep a unified process.
> 
> > driver has no knowledge about what the data is or where/if the data will
> > be stored.
> 
> The fpga-image-load driver knows the data will be stored in nvmem, while
> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> to know the exact physical position to store, may not, depends on what the
> HW engines are.
> 
> > 
> > This functionality could, of course, be merged into the fpga-mgr. I did
> > a proof of concept of this a while back and we discussed the pros and cons.
> > See this email for a recap:
> > 
> > https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> > 
> > Things have changed some with the evolution of the driver. The IOCTL
> > approach probably fits better than the sysfs implementation. At the time
> > it seemed that a merge would add unnecessary complexity without adding value.
> 
> I think at least developers don't have to go through 2 sets of software
> stacks which are of the same concept. And adding some new features like
> optionally threading or canceling data transfer are also good to FPGA
> region update. And the nvmem update could also be benifit from exsiting
> implementations like scatter-gather buffers, in-kernel firmware loading.
> 
> I try to explain myself according to each of your concern from that mail
> thread:
> 
> Purpose of the 2 updates
> ========================
> 
>   As I said before, I think they are both data transfer devices. FPGA
> region update gets extra support from fpga-region & fpga-bridge, and
> FPGA nvmem update could be a standalone fpga-mgr.
> 
> Extra APIs that are unique to nvmem update
> ==========================================
> 
>   cdev APIs for nvmem update:
>     Yes we need to expand the functionality so we need them.
> 
>   available_images, image_load APIs for loading nvmem content to FPGA
>   region:
>     These are features in later patchsets which are not submitted, but we
>     could talk about it in advance. I think this is actually a FPGA region
>     update from nvmem, it also requires gating, data loading (no SW transfer)
>     and re-enumeration, or a single command to image_load HW may result system
>     disordered. The FPGA framework now only supports update from in-kernel
>     user data, maybe we add support for update from nvmem later.
> 
>   fpga-mgr state extend:
>     I think it could be extended, The current states are not perfect,
>     adding something like IDLE or READY is just fine.
> 
>   fpga-mgr status extend:
>     Add general error definitions as needed. If there is some status
>     that is quite vendor specific, expose it in low-level driver.
> 
>   remaining-size:
>     Nice to have for all.
> 
> Threading the update
> ====================
> 
>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>   reprogrammer?

Another thought is, could we implement the non-threading version
firstly, so there would be less change and faster to have the basic
functionality. But either is OK for me.

Thanks,
Yilun

> 
> Cancelling the update
> ====================
> 
>   Also a good option for FPGA region update if HW engine supports.
> 
> Thanks,
> Yilun
> 
> > 
> > >>>>
> > >>>> According to the patchset, the basic workflow of the 2 update types are
> > >>>> quite similar, get the data, prepare for the HW, write and complete.
> > >>>> They are already implemented in FPGA manager. We've discussed some
> > >>>> differences like threading or canceling the update, which are
> > >>>> not provided by FPGA manager but they may also nice to have for FPGA
> > >>>> region update. An FPGA region update may also last for a long time??
> > >>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> > >>>>
> > >>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> > >>>> fpga_image_info to indicate the HW capability (support FPGA region
> > >>>> update or nvmem update or both) of the download engine and the provided
> > >>>> image type. Then the low-level driver knows how to download if it
> > >>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> > >>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> > >>>> update cause it changes the system HW devices and needs device hotplug
> > >>>> in FPGA region. We'd better leave it to FPGA region class, this is
> > >>>> another topic.
> > I'll give this some more thought and see if I can come up with some RFC
> > patches.
> > 
> > - Russ
> > >>>>
> > >>>> More discussion is appreciated.
> > >>> I also think fpga_mgr could be extended.
> > >>>
> > >>> In this patchset,
> > >>>
> > >>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> > >>>
> > >>> A second, similar set of write ops was added to fpga_manger_ops,
> > >>>
> > >>> new bit/flag was added to fpga_image_info
> > >>>
> > >>> The intent was for dfl to add their specific ops to cover what is done in
> > >>> this patchset.
> > >> I think we don't have to add 2 ops for reconfig & reimage in framework,
> > >> the 2 processes are almost the same.
> > >>
> > >> Just add the _REIMAGE (or something else, NVMEM?) flag for
> > >> fpga_image_info, and low level drivers handle it as they do for other
> > >> flags.
> > >>
> > >> How do you think?
> > >
> > > A single set is fine.
> > >
> > > A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> > >
> > > Tom
> > >
> > >>
> > >> Thanks,
> > >> Yilun
> > >>
> > >>> Any other driver would do similar.
> > >>>
> > >>> Is this close to what you are thinking ?
> > >>>
> > >>> Tom
> > >>>
> > >>>> Thanks,
> > >>>> Yilun
> > >>>>
> > >

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/12/21 12:47 AM, Xu Yilun wrote:
> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
>>
>> On 10/11/21 5:35 AM, Tom Rix wrote:
>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>>>> The FPGA Image Load framework provides an API to upload image
>>>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>>>> specific files. It is up to the lower-level device driver and the
>>>>>>> target device to authenticate and disposition the file data.
>>>>>> I've reconsider the FPGA persistent image update again, and think we
>>>>>> may include it in FPGA manager framework.
>>>>>>
>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>>>> I need to consider more seriously than before.
>>>>>>
>>>>>> We have consensus the FPGA persistent image update is just like a normal
>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>>>> while the current FPGA manager deals with the active FPGA region update
>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>>>> supports updating both FPGA region and nvmem.
>> The fpga-image-load driver is actually just a data transfer. The class
> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> acting as the FPGA region admin responsible for gating, transfering and
> re-enumerating.
>
> So my opinion is to add a new data transfer type and keep a unified process.
>
>> driver has no knowledge about what the data is or where/if the data will
>> be stored.
> The fpga-image-load driver knows the data will be stored in nvmem,
FYI: This is not strictly correct. In a coming product there is a
case where the data will be stored in RAM. Richard Gong was also
looking to use this driver to validate an image without programming
or storing it. The fpga-image-load driver has no expectation that
the data will be stored in nvmem, or even that it will be stored
at all.

> while
> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> to know the exact physical position to store, may not, depends on what the
> HW engines are.
>
>> This functionality could, of course, be merged into the fpga-mgr. I did
>> a proof of concept of this a while back and we discussed the pros and cons.
>> See this email for a recap:
>>
>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
>>
>> Things have changed some with the evolution of the driver. The IOCTL
>> approach probably fits better than the sysfs implementation. At the time
>> it seemed that a merge would add unnecessary complexity without adding value.
> I think at least developers don't have to go through 2 sets of software
> stacks which are of the same concept. And adding some new features like
> optionally threading or canceling data transfer are also good to FPGA
> region update. And the nvmem update could also be benifit from exsiting
> implementations like scatter-gather buffers, in-kernel firmware loading.
>
> I try to explain myself according to each of your concern from that mail
> thread:
>
> Purpose of the 2 updates
> ========================
>
>   As I said before, I think they are both data transfer devices. FPGA
> region update gets extra support from fpga-region & fpga-bridge, and
> FPGA nvmem update could be a standalone fpga-mgr.
>
> Extra APIs that are unique to nvmem update
> ==========================================
>
>   cdev APIs for nvmem update:
>     Yes we need to expand the functionality so we need them.
>
>   available_images, image_load APIs for loading nvmem content to FPGA
>   region:
>     These are features in later patchsets which are not submitted, but we
>     could talk about it in advance. I think this is actually a FPGA region
>     update from nvmem, it also requires gating, data loading (no SW transfer)
>     and re-enumeration, or a single command to image_load HW may result system
>     disordered. The FPGA framework now only supports update from in-kernel
>     user data, maybe we add support for update from nvmem later.
>
>   fpga-mgr state extend:
>     I think it could be extended, The current states are not perfect,
>     adding something like IDLE or READY is just fine.
>
>   fpga-mgr status extend:
>     Add general error definitions as needed. If there is some status
>     that is quite vendor specific, expose it in low-level driver.
>
>   remaining-size:
>     Nice to have for all.
>
> Threading the update
> ====================
>
>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>   reprogrammer?
>
> Cancelling the update
> ====================
>
>   Also a good option for FPGA region update if HW engine supports.
These are all good points.

Thanks,
- Russ
>
> Thanks,
> Yilun
>
>>>>>> According to the patchset, the basic workflow of the 2 update types are
>>>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>>>> They are already implemented in FPGA manager. We've discussed some
>>>>>> differences like threading or canceling the update, which are
>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>>>> region update. An FPGA region update may also last for a long time??
>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>>>
>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>>>> update or nvmem update or both) of the download engine and the provided
>>>>>> image type. Then the low-level driver knows how to download if it
>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>>>> update cause it changes the system HW devices and needs device hotplug
>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>>>> another topic.
>> I'll give this some more thought and see if I can come up with some RFC
>> patches.
>>
>> - Russ
>>>>>> More discussion is appreciated.
>>>>> I also think fpga_mgr could be extended.
>>>>>
>>>>> In this patchset,
>>>>>
>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>>>
>>>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>>>
>>>>> new bit/flag was added to fpga_image_info
>>>>>
>>>>> The intent was for dfl to add their specific ops to cover what is done in
>>>>> this patchset.
>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>>>> the 2 processes are almost the same.
>>>>
>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>>>> fpga_image_info, and low level drivers handle it as they do for other
>>>> flags.
>>>>
>>>> How do you think?
>>> A single set is fine.
>>>
>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>>>
>>> Tom
>>>
>>>> Thanks,
>>>> Yilun
>>>>
>>>>> Any other driver would do similar.
>>>>>
>>>>> Is this close to what you are thinking ?
>>>>>
>>>>> Tom
>>>>>
>>>>>> Thanks,
>>>>>> Yilun
>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
> 
> 
> On 10/12/21 12:47 AM, Xu Yilun wrote:
> > On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> >>
> >> On 10/11/21 5:35 AM, Tom Rix wrote:
> >>> On 10/10/21 6:41 PM, Xu Yilun wrote:
> >>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>>>> The FPGA Image Load framework provides an API to upload image
> >>>>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>>>> specific files. It is up to the lower-level device driver and the
> >>>>>>> target device to authenticate and disposition the file data.
> >>>>>> I've reconsider the FPGA persistent image update again, and think we
> >>>>>> may include it in FPGA manager framework.
> >>>>>>
> >>>>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>>>> I need to consider more seriously than before.
> >>>>>>
> >>>>>> We have consensus the FPGA persistent image update is just like a normal
> >>>>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>>>> while the current FPGA manager deals with the active FPGA region update
> >>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>>>> supports updating both FPGA region and nvmem.
> >> The fpga-image-load driver is actually just a data transfer. The class
> > IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> > acting as the FPGA region admin responsible for gating, transfering and
> > re-enumerating.
> >
> > So my opinion is to add a new data transfer type and keep a unified process.
> >
> >> driver has no knowledge about what the data is or where/if the data will
> >> be stored.
> > The fpga-image-load driver knows the data will be stored in nvmem,
> FYI: This is not strictly correct. In a coming product there is a
> case where the data will be stored in RAM. Richard Gong was also
> looking to use this driver to validate an image without programming
> or storing it. The fpga-image-load driver has no expectation that
> the data will be stored in nvmem, or even that it will be stored
> at all.

OK, we can discuss that use case then. But fundamentally a driver should
be clear what it is doing. You may try to extend the FPGA framework to
support nvmem storage, or image validation, but cannot say we feed the
data to any engine undefined by the framework.

Thanks,
Yilun

> 
> > while
> > the fpga-mgr knows the data will be stored in FPGA cells. They may need
> > to know the exact physical position to store, may not, depends on what the
> > HW engines are.
> >
> >> This functionality could, of course, be merged into the fpga-mgr. I did
> >> a proof of concept of this a while back and we discussed the pros and cons.
> >> See this email for a recap:
> >>
> >> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> >>
> >> Things have changed some with the evolution of the driver. The IOCTL
> >> approach probably fits better than the sysfs implementation. At the time
> >> it seemed that a merge would add unnecessary complexity without adding value.
> > I think at least developers don't have to go through 2 sets of software
> > stacks which are of the same concept. And adding some new features like
> > optionally threading or canceling data transfer are also good to FPGA
> > region update. And the nvmem update could also be benifit from exsiting
> > implementations like scatter-gather buffers, in-kernel firmware loading.
> >
> > I try to explain myself according to each of your concern from that mail
> > thread:
> >
> > Purpose of the 2 updates
> > ========================
> >
> >   As I said before, I think they are both data transfer devices. FPGA
> > region update gets extra support from fpga-region & fpga-bridge, and
> > FPGA nvmem update could be a standalone fpga-mgr.
> >
> > Extra APIs that are unique to nvmem update
> > ==========================================
> >
> >   cdev APIs for nvmem update:
> >     Yes we need to expand the functionality so we need them.
> >
> >   available_images, image_load APIs for loading nvmem content to FPGA
> >   region:
> >     These are features in later patchsets which are not submitted, but we
> >     could talk about it in advance. I think this is actually a FPGA region
> >     update from nvmem, it also requires gating, data loading (no SW transfer)
> >     and re-enumeration, or a single command to image_load HW may result system
> >     disordered. The FPGA framework now only supports update from in-kernel
> >     user data, maybe we add support for update from nvmem later.
> >
> >   fpga-mgr state extend:
> >     I think it could be extended, The current states are not perfect,
> >     adding something like IDLE or READY is just fine.
> >
> >   fpga-mgr status extend:
> >     Add general error definitions as needed. If there is some status
> >     that is quite vendor specific, expose it in low-level driver.
> >
> >   remaining-size:
> >     Nice to have for all.
> >
> > Threading the update
> > ====================
> >
> >   Also a good option for FPGA region update, maybe we also have a slow FPGA
> >   reprogrammer?
> >
> > Cancelling the update
> > ====================
> >
> >   Also a good option for FPGA region update if HW engine supports.
> These are all good points.
> 
> Thanks,
> - Russ
> >
> > Thanks,
> > Yilun
> >
> >>>>>> According to the patchset, the basic workflow of the 2 update types are
> >>>>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>>>> They are already implemented in FPGA manager. We've discussed some
> >>>>>> differences like threading or canceling the update, which are
> >>>>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>>>> region update. An FPGA region update may also last for a long time??
> >>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>>>
> >>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>>>> update or nvmem update or both) of the download engine and the provided
> >>>>>> image type. Then the low-level driver knows how to download if it
> >>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>>>> update cause it changes the system HW devices and needs device hotplug
> >>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>>>> another topic.
> >> I'll give this some more thought and see if I can come up with some RFC
> >> patches.
> >>
> >> - Russ
> >>>>>> More discussion is appreciated.
> >>>>> I also think fpga_mgr could be extended.
> >>>>>
> >>>>> In this patchset,
> >>>>>
> >>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>>>
> >>>>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>>>
> >>>>> new bit/flag was added to fpga_image_info
> >>>>>
> >>>>> The intent was for dfl to add their specific ops to cover what is done in
> >>>>> this patchset.
> >>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >>>> the 2 processes are almost the same.
> >>>>
> >>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >>>> fpga_image_info, and low level drivers handle it as they do for other
> >>>> flags.
> >>>>
> >>>> How do you think?
> >>> A single set is fine.
> >>>
> >>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >>>
> >>> Tom
> >>>
> >>>> Thanks,
> >>>> Yilun
> >>>>
> >>>>> Any other driver would do similar.
> >>>>>
> >>>>> Is this close to what you are thinking ?
> >>>>>
> >>>>> Tom
> >>>>>
> >>>>>> Thanks,
> >>>>>> Yilun
> >>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/12/21 6:06 PM, Xu Yilun wrote:
> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
>>
>> On 10/12/21 12:47 AM, Xu Yilun wrote:
>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>>>>>> The FPGA Image Load framework provides an API to upload image
>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>>>>>> specific files. It is up to the lower-level device driver and the
>>>>>>>>> target device to authenticate and disposition the file data.
>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
>>>>>>>> may include it in FPGA manager framework.
>>>>>>>>
>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>>>>>> I need to consider more seriously than before.
>>>>>>>>
>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>>>>>> while the current FPGA manager deals with the active FPGA region update
>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>>>>>> supports updating both FPGA region and nvmem.
>>>> The fpga-image-load driver is actually just a data transfer. The class
>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
>>> acting as the FPGA region admin responsible for gating, transfering and
>>> re-enumerating.
>>>
>>> So my opinion is to add a new data transfer type and keep a unified process.
>>>
>>>> driver has no knowledge about what the data is or where/if the data will
>>>> be stored.
>>> The fpga-image-load driver knows the data will be stored in nvmem,
>> FYI: This is not strictly correct. In a coming product there is a
>> case where the data will be stored in RAM. Richard Gong was also
>> looking to use this driver to validate an image without programming
>> or storing it. The fpga-image-load driver has no expectation that
>> the data will be stored in nvmem, or even that it will be stored
>> at all.
> OK, we can discuss that use case then. But fundamentally a driver should
> be clear what it is doing.

The lower-level driver is, of course, clear what it is doing. And the
FPGA Image Load Framework simply provides a consistent API and manages
a potentially long-running data transfer in the context of a kernel
worker thread.

It sounds like you are saying that that is not "clear enough" in the
context of the FPGA Manager?

The files that are used with Intel PAC devices are self-describing. The
user-space tools, the class driver and the lower-level driver just pass
the data through to the card BMC without any knowledge of the content,
purpose or final destination of the data.

The card BMC will receive signed data, validate it, and process it as a
BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
the n6000, it could also be part of a multi-step process for programming
SDM keys and the data may not be stored permanently.

> You may try to extend the FPGA framework to
> support nvmem storage, or image validation, but cannot say we feed the
> data to any engine undefined by the framework.

I'm not sure what you mean by "feed the data to any engine undefined by the
framework". I think the "engine" is the lower level driver/device that invokes
the fpga_mgr. The lower level driver, of course, is clear what it is doing.
The fpga_mgr cannot control what driver invokes it.

Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
data. Meaning that a self-describing file alone is not acceptable?

Thanks,
- Russ

> Thanks,
> Yilun
>
>>> while
>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
>>> to know the exact physical position to store, may not, depends on what the
>>> HW engines are.
>>>
>>>> This functionality could, of course, be merged into the fpga-mgr. I did
>>>> a proof of concept of this a while back and we discussed the pros and cons.
>>>> See this email for a recap:
>>>>
>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
>>>>
>>>> Things have changed some with the evolution of the driver. The IOCTL
>>>> approach probably fits better than the sysfs implementation. At the time
>>>> it seemed that a merge would add unnecessary complexity without adding value.
>>> I think at least developers don't have to go through 2 sets of software
>>> stacks which are of the same concept. And adding some new features like
>>> optionally threading or canceling data transfer are also good to FPGA
>>> region update. And the nvmem update could also be benifit from exsiting
>>> implementations like scatter-gather buffers, in-kernel firmware loading.
>>>
>>> I try to explain myself according to each of your concern from that mail
>>> thread:
>>>
>>> Purpose of the 2 updates
>>> ========================
>>>
>>>   As I said before, I think they are both data transfer devices. FPGA
>>> region update gets extra support from fpga-region & fpga-bridge, and
>>> FPGA nvmem update could be a standalone fpga-mgr.
>>>
>>> Extra APIs that are unique to nvmem update
>>> ==========================================
>>>
>>>   cdev APIs for nvmem update:
>>>     Yes we need to expand the functionality so we need them.
>>>
>>>   available_images, image_load APIs for loading nvmem content to FPGA
>>>   region:
>>>     These are features in later patchsets which are not submitted, but we
>>>     could talk about it in advance. I think this is actually a FPGA region
>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
>>>     and re-enumeration, or a single command to image_load HW may result system
>>>     disordered. The FPGA framework now only supports update from in-kernel
>>>     user data, maybe we add support for update from nvmem later.
>>>
>>>   fpga-mgr state extend:
>>>     I think it could be extended, The current states are not perfect,
>>>     adding something like IDLE or READY is just fine.
>>>
>>>   fpga-mgr status extend:
>>>     Add general error definitions as needed. If there is some status
>>>     that is quite vendor specific, expose it in low-level driver.
>>>
>>>   remaining-size:
>>>     Nice to have for all.
>>>
>>> Threading the update
>>> ====================
>>>
>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>>>   reprogrammer?
>>>
>>> Cancelling the update
>>> ====================
>>>
>>>   Also a good option for FPGA region update if HW engine supports.
>> These are all good points.
>>
>> Thanks,
>> - Russ
>>> Thanks,
>>> Yilun
>>>
>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>>>>>> They are already implemented in FPGA manager. We've discussed some
>>>>>>>> differences like threading or canceling the update, which are
>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>>>>>> region update. An FPGA region update may also last for a long time??
>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>>>>>
>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>>>>>> update or nvmem update or both) of the download engine and the provided
>>>>>>>> image type. Then the low-level driver knows how to download if it
>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>>>>>> update cause it changes the system HW devices and needs device hotplug
>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>>>>>> another topic.
>>>> I'll give this some more thought and see if I can come up with some RFC
>>>> patches.
>>>>
>>>> - Russ
>>>>>>>> More discussion is appreciated.
>>>>>>> I also think fpga_mgr could be extended.
>>>>>>>
>>>>>>> In this patchset,
>>>>>>>
>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>>>>>
>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>>>>>
>>>>>>> new bit/flag was added to fpga_image_info
>>>>>>>
>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
>>>>>>> this patchset.
>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>>>>>> the 2 processes are almost the same.
>>>>>>
>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>>>>>> fpga_image_info, and low level drivers handle it as they do for other
>>>>>> flags.
>>>>>>
>>>>>> How do you think?
>>>>> A single set is fine.
>>>>>
>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>>>>>
>>>>> Tom
>>>>>
>>>>>> Thanks,
>>>>>> Yilun
>>>>>>
>>>>>>> Any other driver would do similar.
>>>>>>>
>>>>>>> Is this close to what you are thinking ?
>>>>>>>
>>>>>>> Tom
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yilun
>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
> 
> 
> On 10/12/21 6:06 PM, Xu Yilun wrote:
> > On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
> >>
> >> On 10/12/21 12:47 AM, Xu Yilun wrote:
> >>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> >>>> On 10/11/21 5:35 AM, Tom Rix wrote:
> >>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
> >>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>>>>>> The FPGA Image Load framework provides an API to upload image
> >>>>>>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>>>>>> specific files. It is up to the lower-level device driver and the
> >>>>>>>>> target device to authenticate and disposition the file data.
> >>>>>>>> I've reconsider the FPGA persistent image update again, and think we
> >>>>>>>> may include it in FPGA manager framework.
> >>>>>>>>
> >>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>>>>>> I need to consider more seriously than before.
> >>>>>>>>
> >>>>>>>> We have consensus the FPGA persistent image update is just like a normal
> >>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>>>>>> while the current FPGA manager deals with the active FPGA region update
> >>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>>>>>> supports updating both FPGA region and nvmem.
> >>>> The fpga-image-load driver is actually just a data transfer. The class
> >>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> >>> acting as the FPGA region admin responsible for gating, transfering and
> >>> re-enumerating.
> >>>
> >>> So my opinion is to add a new data transfer type and keep a unified process.
> >>>
> >>>> driver has no knowledge about what the data is or where/if the data will
> >>>> be stored.
> >>> The fpga-image-load driver knows the data will be stored in nvmem,
> >> FYI: This is not strictly correct. In a coming product there is a
> >> case where the data will be stored in RAM. Richard Gong was also
> >> looking to use this driver to validate an image without programming
> >> or storing it. The fpga-image-load driver has no expectation that
> >> the data will be stored in nvmem, or even that it will be stored
> >> at all.
> > OK, we can discuss that use case then. But fundamentally a driver should
> > be clear what it is doing.
> 
> The lower-level driver is, of course, clear what it is doing. And the
> FPGA Image Load Framework simply provides a consistent API and manages
> a potentially long-running data transfer in the context of a kernel
> worker thread.
> 
> It sounds like you are saying that that is not "clear enough" in the
> context of the FPGA Manager?
> 
> The files that are used with Intel PAC devices are self-describing. The
> user-space tools, the class driver and the lower-level driver just pass
> the data through to the card BMC without any knowledge of the content,
> purpose or final destination of the data.
> 
> The card BMC will receive signed data, validate it, and process it as a
> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In

I category all these actions as firmware update fully or partially on
persistent storage. The FPGA Manager don't have to know the meaning of
every byte on flash, but it should be aware the firmware is updated and
the card may acts differently with a new firmware. This is the common
working model for most of the FPGA cards so that we implement it in FPGA
manager class. 

> the n6000, it could also be part of a multi-step process for programming
> SDM keys and the data may not be stored permanently.

This is new to me, but seems to be different from firmware update, so lets
think about it again.

> 
> > You may try to extend the FPGA framework to
> > support nvmem storage, or image validation, but cannot say we feed the
> > data to any engine undefined by the framework.
> 
> I'm not sure what you mean by "feed the data to any engine undefined by the
> framework". I think the "engine" is the lower level driver/device that invokes
> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
> The fpga_mgr cannot control what driver invokes it.
> 
> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
> data. Meaning that a self-describing file alone is not acceptable?

The class driver should define a reasonable working model and APIs.
Updating the FPGA backup storage is good to me. But receiving a mystery
box and do whatever it requires is not.

Self-describing file is OK, encryption is OK, but either the class
driver itself, or with the help of the low level driver, should make
sure it works within its scope.

Thanks,
Yilun

> 
> Thanks,
> - Russ
> 
> > Thanks,
> > Yilun
> >
> >>> while
> >>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> >>> to know the exact physical position to store, may not, depends on what the
> >>> HW engines are.
> >>>
> >>>> This functionality could, of course, be merged into the fpga-mgr. I did
> >>>> a proof of concept of this a while back and we discussed the pros and cons.
> >>>> See this email for a recap:
> >>>>
> >>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> >>>>
> >>>> Things have changed some with the evolution of the driver. The IOCTL
> >>>> approach probably fits better than the sysfs implementation. At the time
> >>>> it seemed that a merge would add unnecessary complexity without adding value.
> >>> I think at least developers don't have to go through 2 sets of software
> >>> stacks which are of the same concept. And adding some new features like
> >>> optionally threading or canceling data transfer are also good to FPGA
> >>> region update. And the nvmem update could also be benifit from exsiting
> >>> implementations like scatter-gather buffers, in-kernel firmware loading.
> >>>
> >>> I try to explain myself according to each of your concern from that mail
> >>> thread:
> >>>
> >>> Purpose of the 2 updates
> >>> ========================
> >>>
> >>>   As I said before, I think they are both data transfer devices. FPGA
> >>> region update gets extra support from fpga-region & fpga-bridge, and
> >>> FPGA nvmem update could be a standalone fpga-mgr.
> >>>
> >>> Extra APIs that are unique to nvmem update
> >>> ==========================================
> >>>
> >>>   cdev APIs for nvmem update:
> >>>     Yes we need to expand the functionality so we need them.
> >>>
> >>>   available_images, image_load APIs for loading nvmem content to FPGA
> >>>   region:
> >>>     These are features in later patchsets which are not submitted, but we
> >>>     could talk about it in advance. I think this is actually a FPGA region
> >>>     update from nvmem, it also requires gating, data loading (no SW transfer)
> >>>     and re-enumeration, or a single command to image_load HW may result system
> >>>     disordered. The FPGA framework now only supports update from in-kernel
> >>>     user data, maybe we add support for update from nvmem later.
> >>>
> >>>   fpga-mgr state extend:
> >>>     I think it could be extended, The current states are not perfect,
> >>>     adding something like IDLE or READY is just fine.
> >>>
> >>>   fpga-mgr status extend:
> >>>     Add general error definitions as needed. If there is some status
> >>>     that is quite vendor specific, expose it in low-level driver.
> >>>
> >>>   remaining-size:
> >>>     Nice to have for all.
> >>>
> >>> Threading the update
> >>> ====================
> >>>
> >>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
> >>>   reprogrammer?
> >>>
> >>> Cancelling the update
> >>> ====================
> >>>
> >>>   Also a good option for FPGA region update if HW engine supports.
> >> These are all good points.
> >>
> >> Thanks,
> >> - Russ
> >>> Thanks,
> >>> Yilun
> >>>
> >>>>>>>> According to the patchset, the basic workflow of the 2 update types are
> >>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>>>>>> They are already implemented in FPGA manager. We've discussed some
> >>>>>>>> differences like threading or canceling the update, which are
> >>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>>>>>> region update. An FPGA region update may also last for a long time??
> >>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>>>>>
> >>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>>>>>> update or nvmem update or both) of the download engine and the provided
> >>>>>>>> image type. Then the low-level driver knows how to download if it
> >>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>>>>>> update cause it changes the system HW devices and needs device hotplug
> >>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>>>>>> another topic.
> >>>> I'll give this some more thought and see if I can come up with some RFC
> >>>> patches.
> >>>>
> >>>> - Russ
> >>>>>>>> More discussion is appreciated.
> >>>>>>> I also think fpga_mgr could be extended.
> >>>>>>>
> >>>>>>> In this patchset,
> >>>>>>>
> >>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>>>>>
> >>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>>>>>
> >>>>>>> new bit/flag was added to fpga_image_info
> >>>>>>>
> >>>>>>> The intent was for dfl to add their specific ops to cover what is done in
> >>>>>>> this patchset.
> >>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >>>>>> the 2 processes are almost the same.
> >>>>>>
> >>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >>>>>> fpga_image_info, and low level drivers handle it as they do for other
> >>>>>> flags.
> >>>>>>
> >>>>>> How do you think?
> >>>>> A single set is fine.
> >>>>>
> >>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >>>>>
> >>>>> Tom
> >>>>>
> >>>>>> Thanks,
> >>>>>> Yilun
> >>>>>>
> >>>>>>> Any other driver would do similar.
> >>>>>>>
> >>>>>>> Is this close to what you are thinking ?
> >>>>>>>
> >>>>>>> Tom
> >>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Yilun
> >>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
> 
> 
> On 10/13/21 6:49 PM, Xu Yilun wrote:
> > On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
> >>
> >> On 10/12/21 6:06 PM, Xu Yilun wrote:
> >>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
> >>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
> >>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> >>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
> >>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
> >>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
> >>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>>>>>>>> specific files. It is up to the lower-level device driver and the
> >>>>>>>>>>> target device to authenticate and disposition the file data.
> >>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
> >>>>>>>>>> may include it in FPGA manager framework.
> >>>>>>>>>>
> >>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>>>>>>>> I need to consider more seriously than before.
> >>>>>>>>>>
> >>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
> >>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
> >>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>>>>>>>> supports updating both FPGA region and nvmem.
> >>>>>> The fpga-image-load driver is actually just a data transfer. The class
> >>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> >>>>> acting as the FPGA region admin responsible for gating, transfering and
> >>>>> re-enumerating.
> >>>>>
> >>>>> So my opinion is to add a new data transfer type and keep a unified process.
> >>>>>
> >>>>>> driver has no knowledge about what the data is or where/if the data will
> >>>>>> be stored.
> >>>>> The fpga-image-load driver knows the data will be stored in nvmem,
> >>>> FYI: This is not strictly correct. In a coming product there is a
> >>>> case where the data will be stored in RAM. Richard Gong was also
> >>>> looking to use this driver to validate an image without programming
> >>>> or storing it. The fpga-image-load driver has no expectation that
> >>>> the data will be stored in nvmem, or even that it will be stored
> >>>> at all.
> >>> OK, we can discuss that use case then. But fundamentally a driver should
> >>> be clear what it is doing.
> >> The lower-level driver is, of course, clear what it is doing. And the
> >> FPGA Image Load Framework simply provides a consistent API and manages
> >> a potentially long-running data transfer in the context of a kernel
> >> worker thread.
> >>
> >> It sounds like you are saying that that is not "clear enough" in the
> >> context of the FPGA Manager?
> >>
> >> The files that are used with Intel PAC devices are self-describing. The
> >> user-space tools, the class driver and the lower-level driver just pass
> >> the data through to the card BMC without any knowledge of the content,
> >> purpose or final destination of the data.
> >>
> >> The card BMC will receive signed data, validate it, and process it as a
> >> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
> > I category all these actions as firmware update fully or partially on
> > persistent storage. The FPGA Manager don't have to know the meaning of
> > every byte on flash, but it should be aware the firmware is updated and
> > the card may acts differently with a new firmware. This is the common
> > working model for most of the FPGA cards so that we implement it in FPGA
> > manager class. 
> >
> >> the n6000, it could also be part of a multi-step process for programming
> >> SDM keys and the data may not be stored permanently.
> > This is new to me, but seems to be different from firmware update, so lets
> > think about it again.
> >
> >>> You may try to extend the FPGA framework to
> >>> support nvmem storage, or image validation, but cannot say we feed the
> >>> data to any engine undefined by the framework.
> >> I'm not sure what you mean by "feed the data to any engine undefined by the
> >> framework". I think the "engine" is the lower level driver/device that invokes
> >> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
> >> The fpga_mgr cannot control what driver invokes it.
> >>
> >> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
> >> data. Meaning that a self-describing file alone is not acceptable?
> > The class driver should define a reasonable working model and APIs.
> > Updating the FPGA backup storage is good to me. But receiving a mystery
> > box and do whatever it requires is not.
> >
> > Self-describing file is OK, encryption is OK, but either the class
> > driver itself, or with the help of the low level driver, should make
> > sure it works within its scope.
> In our secure update process, the card BMC firmware authenticates
> the data using the root entry hashes and will either reject the
> data or perform some function based on the contents. Neither the
> user-space, the class driver, nor the lower level driver know
> what the contents are. It _is_ a "mystery box" to them. How do we
> verify scope in this model?

I think we need to find out how. One case is, the HW is designed to have
one single function, such as firmware update, then any image input
through firmware update API is within expectation, and the driver
should only serve the firmware update API. I think this is how the
N3000 is working now. If the HW is for another function, register itself
to serve another API, or another class driver.

Another case is, the HW could do multiple types of tasks depending on
the content of the image, such as firmware update, image verification,
or assumably power off the card ... There should be some mechanism for
the driver to only accept the right image according to what API is called.
Or the user may input an image named update_the_card.img through
firmware update API and finally get the card off. Having some headers
readable by host for the operation type? Or, some HW interface for host
to apply the operation type as well as the image, let the HW verify?
Let's think about it.

> 
> As you have noted, most current cases result in a change to the
> card, and I suspect that it will remain that way. But that can't be
> guaranteed, and I'm not convinced that a host driver needs to be
> concerned about it.

A host driver should know what is done, in some abstraction level.
I think updating the persistent storage is an acceptable abstraction
in FPGA domain, so I'd like to extend it in FPGA manager. But doing
anything according to the image is not.

Thanks,
Yilun

> 
> - Russ
> 
> >
> > Thanks,
> > Yilun
> >
> >> Thanks,
> >> - Russ
> >>
> >>> Thanks,
> >>> Yilun
> >>>
> >>>>> while
> >>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> >>>>> to know the exact physical position to store, may not, depends on what the
> >>>>> HW engines are.
> >>>>>
> >>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
> >>>>>> a proof of concept of this a while back and we discussed the pros and cons.
> >>>>>> See this email for a recap:
> >>>>>>
> >>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> >>>>>>
> >>>>>> Things have changed some with the evolution of the driver. The IOCTL
> >>>>>> approach probably fits better than the sysfs implementation. At the time
> >>>>>> it seemed that a merge would add unnecessary complexity without adding value.
> >>>>> I think at least developers don't have to go through 2 sets of software
> >>>>> stacks which are of the same concept. And adding some new features like
> >>>>> optionally threading or canceling data transfer are also good to FPGA
> >>>>> region update. And the nvmem update could also be benifit from exsiting
> >>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
> >>>>>
> >>>>> I try to explain myself according to each of your concern from that mail
> >>>>> thread:
> >>>>>
> >>>>> Purpose of the 2 updates
> >>>>> ========================
> >>>>>
> >>>>>   As I said before, I think they are both data transfer devices. FPGA
> >>>>> region update gets extra support from fpga-region & fpga-bridge, and
> >>>>> FPGA nvmem update could be a standalone fpga-mgr.
> >>>>>
> >>>>> Extra APIs that are unique to nvmem update
> >>>>> ==========================================
> >>>>>
> >>>>>   cdev APIs for nvmem update:
> >>>>>     Yes we need to expand the functionality so we need them.
> >>>>>
> >>>>>   available_images, image_load APIs for loading nvmem content to FPGA
> >>>>>   region:
> >>>>>     These are features in later patchsets which are not submitted, but we
> >>>>>     could talk about it in advance. I think this is actually a FPGA region
> >>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
> >>>>>     and re-enumeration, or a single command to image_load HW may result system
> >>>>>     disordered. The FPGA framework now only supports update from in-kernel
> >>>>>     user data, maybe we add support for update from nvmem later.
> >>>>>
> >>>>>   fpga-mgr state extend:
> >>>>>     I think it could be extended, The current states are not perfect,
> >>>>>     adding something like IDLE or READY is just fine.
> >>>>>
> >>>>>   fpga-mgr status extend:
> >>>>>     Add general error definitions as needed. If there is some status
> >>>>>     that is quite vendor specific, expose it in low-level driver.
> >>>>>
> >>>>>   remaining-size:
> >>>>>     Nice to have for all.
> >>>>>
> >>>>> Threading the update
> >>>>> ====================
> >>>>>
> >>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
> >>>>>   reprogrammer?
> >>>>>
> >>>>> Cancelling the update
> >>>>> ====================
> >>>>>
> >>>>>   Also a good option for FPGA region update if HW engine supports.
> >>>> These are all good points.
> >>>>
> >>>> Thanks,
> >>>> - Russ
> >>>>> Thanks,
> >>>>> Yilun
> >>>>>
> >>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
> >>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
> >>>>>>>>>> differences like threading or canceling the update, which are
> >>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>>>>>>>> region update. An FPGA region update may also last for a long time??
> >>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>>>>>>>
> >>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>>>>>>>> update or nvmem update or both) of the download engine and the provided
> >>>>>>>>>> image type. Then the low-level driver knows how to download if it
> >>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
> >>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>>>>>>>> another topic.
> >>>>>> I'll give this some more thought and see if I can come up with some RFC
> >>>>>> patches.
> >>>>>>
> >>>>>> - Russ
> >>>>>>>>>> More discussion is appreciated.
> >>>>>>>>> I also think fpga_mgr could be extended.
> >>>>>>>>>
> >>>>>>>>> In this patchset,
> >>>>>>>>>
> >>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>>>>>>>
> >>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>>>>>>>
> >>>>>>>>> new bit/flag was added to fpga_image_info
> >>>>>>>>>
> >>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
> >>>>>>>>> this patchset.
> >>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >>>>>>>> the 2 processes are almost the same.
> >>>>>>>>
> >>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
> >>>>>>>> flags.
> >>>>>>>>
> >>>>>>>> How do you think?
> >>>>>>> A single set is fine.
> >>>>>>>
> >>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >>>>>>>
> >>>>>>> Tom
> >>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Yilun
> >>>>>>>>
> >>>>>>>>> Any other driver would do similar.
> >>>>>>>>>
> >>>>>>>>> Is this close to what you are thinking ?
> >>>>>>>>>
> >>>>>>>>> Tom
> >>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Yilun
> >>>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/14/21 7:51 PM, Xu Yilun wrote:
> On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
>>
>> On 10/13/21 6:49 PM, Xu Yilun wrote:
>>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
>>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
>>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
>>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
>>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
>>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
>>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
>>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
>>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
>>>>>>>>>>>>> target device to authenticate and disposition the file data.
>>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
>>>>>>>>>>>> may include it in FPGA manager framework.
>>>>>>>>>>>>
>>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>>>>>>>>>> I need to consider more seriously than before.
>>>>>>>>>>>>
>>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
>>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
>>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>>>>>>>>>> supports updating both FPGA region and nvmem.
>>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
>>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
>>>>>>> acting as the FPGA region admin responsible for gating, transfering and
>>>>>>> re-enumerating.
>>>>>>>
>>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
>>>>>>>
>>>>>>>> driver has no knowledge about what the data is or where/if the data will
>>>>>>>> be stored.
>>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
>>>>>> FYI: This is not strictly correct. In a coming product there is a
>>>>>> case where the data will be stored in RAM. Richard Gong was also
>>>>>> looking to use this driver to validate an image without programming
>>>>>> or storing it. The fpga-image-load driver has no expectation that
>>>>>> the data will be stored in nvmem, or even that it will be stored
>>>>>> at all.
>>>>> OK, we can discuss that use case then. But fundamentally a driver should
>>>>> be clear what it is doing.
>>>> The lower-level driver is, of course, clear what it is doing. And the
>>>> FPGA Image Load Framework simply provides a consistent API and manages
>>>> a potentially long-running data transfer in the context of a kernel
>>>> worker thread.
>>>>
>>>> It sounds like you are saying that that is not "clear enough" in the
>>>> context of the FPGA Manager?
>>>>
>>>> The files that are used with Intel PAC devices are self-describing. The
>>>> user-space tools, the class driver and the lower-level driver just pass
>>>> the data through to the card BMC without any knowledge of the content,
>>>> purpose or final destination of the data.
>>>>
>>>> The card BMC will receive signed data, validate it, and process it as a
>>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
>>> I category all these actions as firmware update fully or partially on
>>> persistent storage. The FPGA Manager don't have to know the meaning of
>>> every byte on flash, but it should be aware the firmware is updated and
>>> the card may acts differently with a new firmware. This is the common
>>> working model for most of the FPGA cards so that we implement it in FPGA
>>> manager class. 
>>>
>>>> the n6000, it could also be part of a multi-step process for programming
>>>> SDM keys and the data may not be stored permanently.
>>> This is new to me, but seems to be different from firmware update, so lets
>>> think about it again.
>>>
>>>>> You may try to extend the FPGA framework to
>>>>> support nvmem storage, or image validation, but cannot say we feed the
>>>>> data to any engine undefined by the framework.
>>>> I'm not sure what you mean by "feed the data to any engine undefined by the
>>>> framework". I think the "engine" is the lower level driver/device that invokes
>>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
>>>> The fpga_mgr cannot control what driver invokes it.
>>>>
>>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
>>>> data. Meaning that a self-describing file alone is not acceptable?
>>> The class driver should define a reasonable working model and APIs.
>>> Updating the FPGA backup storage is good to me. But receiving a mystery
>>> box and do whatever it requires is not.
>>>
>>> Self-describing file is OK, encryption is OK, but either the class
>>> driver itself, or with the help of the low level driver, should make
>>> sure it works within its scope.
>> In our secure update process, the card BMC firmware authenticates
>> the data using the root entry hashes and will either reject the
>> data or perform some function based on the contents. Neither the
>> user-space, the class driver, nor the lower level driver know
>> what the contents are. It _is_ a "mystery box" to them. How do we
>> verify scope in this model?
> I think we need to find out how. One case is, the HW is designed to have
> one single function, such as firmware update, then any image input
> through firmware update API is within expectation, and the driver
> should only serve the firmware update API. I think this is how the
> N3000 is working now. If the HW is for another function, register itself
> to serve another API, or another class driver.
>
> Another case is, the HW could do multiple types of tasks depending on
> the content of the image, such as firmware update, image verification,
> or assumably power off the card ... There should be some mechanism for
> the driver to only accept the right image according to what API is called.
> Or the user may input an image named update_the_card.img through
> firmware update API and finally get the card off. Having some headers
> readable by host for the operation type? Or, some HW interface for host
> to apply the operation type as well as the image, let the HW verify?
> Let's think about it.
I'm not sure if I am following your thinking here. The HW, of course,
verifies authentication of the image and acts according to the image
type. I don't think it is reasonable to require the class driver to
interpret the data to determine what it is. That implies that the
class driver would have to know the header format and possible values.
It also means that changes to the header format would require patches
to the class driver.

The FPGA card is trusted by virtue of the fact that the customer
purchased it and physically placed it in the machine. If the FPGA card
(or the lower level driver) validates the image, then why does the
Class driver need to be concerned about the file type? I think the
purpose of the class driver is primarily to provide a common API and
perform common functions so that they don't have to be replicated
among similar low-level drivers. It is up to the low-level driver
or the device itself to ensure that the data received is acceptable.

If the card receives data through the fpga-mgr upload facility that
isn't strictly a firmware update, and if the lower-level driver or
the card handles it and returns appropriate status - is that really
a problem?
>> As you have noted, most current cases result in a change to the
>> card, and I suspect that it will remain that way. But that can't be
>> guaranteed, and I'm not convinced that a host driver needs to be
>> concerned about it.
> A host driver should know what is done, in some abstraction level.
> I think updating the persistent storage is an acceptable abstraction
> in FPGA domain, so I'd like to extend it in FPGA manager. But doing
> anything according to the image is not.
By host driver, do you mean the class driver? Or the lower-level device
driver?

It seems to me that you are saying that self-describing images are not
acceptable? Or at least they are not acceptable payload for an FPGA
Manager firmware-update API?

The FPGA Image Load Framework was designed with the concept of
transferring data to a device without imposing a purpose on the data.
The expectation is that the lower-level driver or the device will
validate the data. Is there something fundamentally wrong with that
approach? And if not, why couldn't we incorporate a similar image_load
API into the FPGA Manager?

- Russ

>
> Thanks,
> Yilun
>
>> - Russ
>>
>>> Thanks,
>>> Yilun
>>>
>>>> Thanks,
>>>> - Russ
>>>>
>>>>> Thanks,
>>>>> Yilun
>>>>>
>>>>>>> while
>>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
>>>>>>> to know the exact physical position to store, may not, depends on what the
>>>>>>> HW engines are.
>>>>>>>
>>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
>>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
>>>>>>>> See this email for a recap:
>>>>>>>>
>>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
>>>>>>>>
>>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
>>>>>>>> approach probably fits better than the sysfs implementation. At the time
>>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
>>>>>>> I think at least developers don't have to go through 2 sets of software
>>>>>>> stacks which are of the same concept. And adding some new features like
>>>>>>> optionally threading or canceling data transfer are also good to FPGA
>>>>>>> region update. And the nvmem update could also be benifit from exsiting
>>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
>>>>>>>
>>>>>>> I try to explain myself according to each of your concern from that mail
>>>>>>> thread:
>>>>>>>
>>>>>>> Purpose of the 2 updates
>>>>>>> ========================
>>>>>>>
>>>>>>>   As I said before, I think they are both data transfer devices. FPGA
>>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
>>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
>>>>>>>
>>>>>>> Extra APIs that are unique to nvmem update
>>>>>>> ==========================================
>>>>>>>
>>>>>>>   cdev APIs for nvmem update:
>>>>>>>     Yes we need to expand the functionality so we need them.
>>>>>>>
>>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
>>>>>>>   region:
>>>>>>>     These are features in later patchsets which are not submitted, but we
>>>>>>>     could talk about it in advance. I think this is actually a FPGA region
>>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
>>>>>>>     and re-enumeration, or a single command to image_load HW may result system
>>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
>>>>>>>     user data, maybe we add support for update from nvmem later.
>>>>>>>
>>>>>>>   fpga-mgr state extend:
>>>>>>>     I think it could be extended, The current states are not perfect,
>>>>>>>     adding something like IDLE or READY is just fine.
>>>>>>>
>>>>>>>   fpga-mgr status extend:
>>>>>>>     Add general error definitions as needed. If there is some status
>>>>>>>     that is quite vendor specific, expose it in low-level driver.
>>>>>>>
>>>>>>>   remaining-size:
>>>>>>>     Nice to have for all.
>>>>>>>
>>>>>>> Threading the update
>>>>>>> ====================
>>>>>>>
>>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>>>>>>>   reprogrammer?
>>>>>>>
>>>>>>> Cancelling the update
>>>>>>> ====================
>>>>>>>
>>>>>>>   Also a good option for FPGA region update if HW engine supports.
>>>>>> These are all good points.
>>>>>>
>>>>>> Thanks,
>>>>>> - Russ
>>>>>>> Thanks,
>>>>>>> Yilun
>>>>>>>
>>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
>>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
>>>>>>>>>>>> differences like threading or canceling the update, which are
>>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
>>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>>>>>>>>>
>>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
>>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
>>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
>>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>>>>>>>>>> another topic.
>>>>>>>> I'll give this some more thought and see if I can come up with some RFC
>>>>>>>> patches.
>>>>>>>>
>>>>>>>> - Russ
>>>>>>>>>>>> More discussion is appreciated.
>>>>>>>>>>> I also think fpga_mgr could be extended.
>>>>>>>>>>>
>>>>>>>>>>> In this patchset,
>>>>>>>>>>>
>>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>>>>>>>>>
>>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>>>>>>>>>
>>>>>>>>>>> new bit/flag was added to fpga_image_info
>>>>>>>>>>>
>>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
>>>>>>>>>>> this patchset.
>>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>>>>>>>>>> the 2 processes are almost the same.
>>>>>>>>>>
>>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
>>>>>>>>>> flags.
>>>>>>>>>>
>>>>>>>>>> How do you think?
>>>>>>>>> A single set is fine.
>>>>>>>>>
>>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>>>>>>>>>
>>>>>>>>> Tom
>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Yilun
>>>>>>>>>>
>>>>>>>>>>> Any other driver would do similar.
>>>>>>>>>>>
>>>>>>>>>>> Is this close to what you are thinking ?
>>>>>>>>>>>
>>>>>>>>>>> Tom
>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Yilun
>>>>>>>>>>>>

Re: [PATCH v17 4/5] fpga: image-load: add status ioctl

[Lizhi Hou]

On 9/29/21 4:00 PM, Russ Weight wrote:
> Extend the FPGA Image Load framework to include an FPGA_IMAGE_LOAD_STATUS
> IOCTL that can be used to monitor the progress of an ongoing image upload.
> The status returned includes how much data remains to be transferred, the
> progress of the image upload, and error information in the case of a
> failure.
>
> Signed-off-by: Russ Weight <russell.h.weight@intel.com>
> ---
> v17:
>   - Rebased for changes to earlier patches.
> v16:
>   - Minor changes to adapt in changes in prevoius patches.
> v15:
>   - This patch is new to the patchset and provides an FPGA_IMAGE_LOAD_STATUS
>     IOCTL to return the current values for: remaining_size, progress,
>     err_progress, and err_code.
>   - This patch has elements of the following three patches from the previous
>     patch-set:
>       [PATCH v14 3/6] fpga: sec-mgr: expose sec-mgr update status
>       [PATCH v14 4/6] fpga: sec-mgr: expose sec-mgr update errors
>       [PATCH v14 5/6] fpga: sec-mgr: expose sec-mgr update size
>   - Changed file, symbol, and config names to reflect the new driver name
>   - There are some minor changes to locking to enable this ioctl to return
>     coherent data.
> ---
>   Documentation/fpga/fpga-image-load.rst |  6 +++
>   drivers/fpga/fpga-image-load.c         | 58 +++++++++++++++++++++-----
>   include/linux/fpga/fpga-image-load.h   |  1 +
>   include/uapi/linux/fpga-image-load.h   | 18 ++++++++
>   4 files changed, 73 insertions(+), 10 deletions(-)
>
> diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
> index 487b5466f67c..f64f5ee473b8 100644
> --- a/Documentation/fpga/fpga-image-load.rst
> +++ b/Documentation/fpga/fpga-image-load.rst
> @@ -33,3 +33,9 @@ being updated. This is an exclusive operation; an attempt to start
>   concurrent image uploads for the same device will fail with EBUSY. An
>   eventfd file descriptor parameter is provided to this IOCTL. It will be
>   signalled at the completion of the image upload.
> +
> +FPGA_IMAGE_LOAD_STATUS:
> +
> +Collect status for an on-going image upload. The status returned includes
> +how much data remains to be transferred, the progress of the image upload,
> +and error information in the case of a failure.
> diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
> index f04dfc71c190..58373b9e8c02 100644
> --- a/drivers/fpga/fpga-image-load.c
> +++ b/drivers/fpga/fpga-image-load.c
> @@ -22,6 +22,22 @@ static dev_t fpga_image_devt;
>
>   #define to_image_load(d) container_of(d, struct fpga_image_load, dev)
>
> +static void fpga_image_update_progress(struct fpga_image_load *imgld,
> +                                      u32 new_progress)
> +{
> +       mutex_lock(&imgld->lock);
> +       imgld->progress = new_progress;
> +       mutex_unlock(&imgld->lock);
> +}
> +
> +static void fpga_image_set_error(struct fpga_image_load *imgld, u32 err_code)
> +{
> +       mutex_lock(&imgld->lock);
> +       imgld->err_progress = imgld->progress;
> +       imgld->err_code = err_code;
> +       mutex_unlock(&imgld->lock);
> +}
> +
>   static void fpga_image_prog_complete(struct fpga_image_load *imgld)
>   {
>          mutex_lock(&imgld->lock);
> @@ -38,24 +54,24 @@ static void fpga_image_do_load(struct work_struct *work)
>          imgld = container_of(work, struct fpga_image_load, work);
>
>          if (imgld->driver_unload) {
> -               imgld->err_code = FPGA_IMAGE_ERR_CANCELED;
> +               fpga_image_set_error(imgld, FPGA_IMAGE_ERR_CANCELED);
>                  goto idle_exit;
>          }
>
>          get_device(&imgld->dev);
>          if (!try_module_get(imgld->dev.parent->driver->owner)) {
> -               imgld->err_code = FPGA_IMAGE_ERR_BUSY;
> +               fpga_image_set_error(imgld, FPGA_IMAGE_ERR_BUSY);
>                  goto putdev_exit;
>          }
>
> -       imgld->progress = FPGA_IMAGE_PROG_PREPARING;
> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PREPARING);
>          ret = imgld->ops->prepare(imgld, imgld->data, imgld->remaining_size);
>          if (ret) {
> -               imgld->err_code = ret;
> +               fpga_image_set_error(imgld, ret);
>                  goto modput_exit;
>          }
>
> -       imgld->progress = FPGA_IMAGE_PROG_WRITING;
> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_WRITING);
>          while (imgld->remaining_size) {
>                  ret = imgld->ops->write(imgld, imgld->data, offset,
>                                          imgld->remaining_size);
> @@ -65,7 +81,7 @@ static void fpga_image_do_load(struct work_struct *work)
>                                           "write-op wrote zero data\n");
>                                  ret = -FPGA_IMAGE_ERR_RW_ERROR;
>                          }
> -                       imgld->err_code = -ret;
> +                       fpga_image_set_error(imgld, -ret);
>                          goto done;
>                  }
>
> @@ -73,10 +89,10 @@ static void fpga_image_do_load(struct work_struct *work)
>                  offset += ret;
>          }
>
> -       imgld->progress = FPGA_IMAGE_PROG_PROGRAMMING;
> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PROGRAMMING);
>          ret = imgld->ops->poll_complete(imgld);
>          if (ret)
> -               imgld->err_code = ret;
> +               fpga_image_set_error(imgld, ret);
>
>   done:
>          if (imgld->ops->cleanup)
> @@ -151,20 +167,42 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
>          return ret;
>   }
>
> +static int fpga_image_load_ioctl_status(struct fpga_image_load *imgld,
> +                                       unsigned long arg)
> +{
> +       struct fpga_image_status status;
> +
> +       memset(&status, 0, sizeof(status));
> +       status.progress = imgld->progress;
> +       status.remaining_size = imgld->remaining_size;
> +       status.err_progress = imgld->err_progress;
> +       status.err_code = imgld->err_code;
> +
> +       if (copy_to_user((void __user *)arg, &status, sizeof(status)))
> +               return -EFAULT;
> +
> +       return 0;
> +}
> +
>   static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
>                                    unsigned long arg)
>   {
>          struct fpga_image_load *imgld = filp->private_data;
>          int ret = -ENOTTY;
>
> +       mutex_lock(&imgld->lock);
> +
>          switch (cmd) {
>          case FPGA_IMAGE_LOAD_WRITE:
> -               mutex_lock(&imgld->lock);
>                  ret = fpga_image_load_ioctl_write(imgld, arg);
> -               mutex_unlock(&imgld->lock);
> +               break;
> +       case FPGA_IMAGE_LOAD_STATUS:
> +               ret = fpga_image_load_ioctl_status(imgld, arg);
>                  break;
>          }
>
> +       mutex_unlock(&imgld->lock);
> +
>          return ret;
>   }
>
> diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
> index 77b3c91ce073..366111d090fb 100644
> --- a/include/linux/fpga/fpga-image-load.h
> +++ b/include/linux/fpga/fpga-image-load.h
> @@ -49,6 +49,7 @@ struct fpga_image_load {
>          const u8 *data;                 /* pointer to update data */
>          u32 remaining_size;             /* size remaining to transfer */
>          u32 progress;
> +       u32 err_progress;               /* progress at time of error */
>          u32 err_code;                   /* image load error code */
>          bool driver_unload;
>          struct eventfd_ctx *finished;
> diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
> index 8d2d3db92e87..1b91343961df 100644
> --- a/include/uapi/linux/fpga-image-load.h
> +++ b/include/uapi/linux/fpga-image-load.h
> @@ -51,4 +51,22 @@ struct fpga_image_write {
>
>   #define FPGA_IMAGE_LOAD_WRITE  _IOW(FPGA_IMAGE_LOAD_MAGIC, 0, struct fpga_image_write)
>
> +/**
> + * FPGA_IMAGE_LOAD_STATUS - _IOR(FPGA_IMAGE_LOAD_MAGIC, 1,
> + *                              struct fpga_image_status)
> + *
> + * Request status information for an ongoing update.
> + *
> + * Return: 0 on success, -errno on failure.
> + */
> +struct fpga_image_status {
> +       /* Output */
> +       __u32 remaining_size;   /* size remaining to transfer */
> +       __u32 progress;         /* current progress of image load */
> +       __u32 err_progress;     /* progress at time of error */
> +       __u32 err_code;         /* error code */
> +};

Could this be extended to also collect the image detail?

         Image version, name, etc been successfully written to device 
(flash).

         Image version, name, etc is currently running on the device.

Or maybe add another query command to do these?

So the userland utility will be able to show what image is running and 
what image is going to run with next cold reboot.


Thanks,

Lizhi

> +
> +#define FPGA_IMAGE_LOAD_STATUS _IOR(FPGA_IMAGE_LOAD_MAGIC, 1, struct fpga_image_status)
> +
>   #endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */
> --
> 2.25.1
>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Fri, Oct 15, 2021 at 10:34:23AM -0700, Russ Weight wrote:
> 
> 
> On 10/14/21 7:51 PM, Xu Yilun wrote:
> > On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
> >>
> >> On 10/13/21 6:49 PM, Xu Yilun wrote:
> >>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
> >>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
> >>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
> >>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
> >>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> >>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
> >>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
> >>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
> >>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
> >>>>>>>>>>>>> target device to authenticate and disposition the file data.
> >>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
> >>>>>>>>>>>> may include it in FPGA manager framework.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>>>>>>>>>> I need to consider more seriously than before.
> >>>>>>>>>>>>
> >>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
> >>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
> >>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>>>>>>>>>> supports updating both FPGA region and nvmem.
> >>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
> >>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> >>>>>>> acting as the FPGA region admin responsible for gating, transfering and
> >>>>>>> re-enumerating.
> >>>>>>>
> >>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
> >>>>>>>
> >>>>>>>> driver has no knowledge about what the data is or where/if the data will
> >>>>>>>> be stored.
> >>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
> >>>>>> FYI: This is not strictly correct. In a coming product there is a
> >>>>>> case where the data will be stored in RAM. Richard Gong was also
> >>>>>> looking to use this driver to validate an image without programming
> >>>>>> or storing it. The fpga-image-load driver has no expectation that
> >>>>>> the data will be stored in nvmem, or even that it will be stored
> >>>>>> at all.
> >>>>> OK, we can discuss that use case then. But fundamentally a driver should
> >>>>> be clear what it is doing.
> >>>> The lower-level driver is, of course, clear what it is doing. And the
> >>>> FPGA Image Load Framework simply provides a consistent API and manages
> >>>> a potentially long-running data transfer in the context of a kernel
> >>>> worker thread.
> >>>>
> >>>> It sounds like you are saying that that is not "clear enough" in the
> >>>> context of the FPGA Manager?
> >>>>
> >>>> The files that are used with Intel PAC devices are self-describing. The
> >>>> user-space tools, the class driver and the lower-level driver just pass
> >>>> the data through to the card BMC without any knowledge of the content,
> >>>> purpose or final destination of the data.
> >>>>
> >>>> The card BMC will receive signed data, validate it, and process it as a
> >>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
> >>> I category all these actions as firmware update fully or partially on
> >>> persistent storage. The FPGA Manager don't have to know the meaning of
> >>> every byte on flash, but it should be aware the firmware is updated and
> >>> the card may acts differently with a new firmware. This is the common
> >>> working model for most of the FPGA cards so that we implement it in FPGA
> >>> manager class. 
> >>>
> >>>> the n6000, it could also be part of a multi-step process for programming
> >>>> SDM keys and the data may not be stored permanently.
> >>> This is new to me, but seems to be different from firmware update, so lets
> >>> think about it again.
> >>>
> >>>>> You may try to extend the FPGA framework to
> >>>>> support nvmem storage, or image validation, but cannot say we feed the
> >>>>> data to any engine undefined by the framework.
> >>>> I'm not sure what you mean by "feed the data to any engine undefined by the
> >>>> framework". I think the "engine" is the lower level driver/device that invokes
> >>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
> >>>> The fpga_mgr cannot control what driver invokes it.
> >>>>
> >>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
> >>>> data. Meaning that a self-describing file alone is not acceptable?
> >>> The class driver should define a reasonable working model and APIs.
> >>> Updating the FPGA backup storage is good to me. But receiving a mystery
> >>> box and do whatever it requires is not.
> >>>
> >>> Self-describing file is OK, encryption is OK, but either the class
> >>> driver itself, or with the help of the low level driver, should make
> >>> sure it works within its scope.
> >> In our secure update process, the card BMC firmware authenticates
> >> the data using the root entry hashes and will either reject the
> >> data or perform some function based on the contents. Neither the
> >> user-space, the class driver, nor the lower level driver know
> >> what the contents are. It _is_ a "mystery box" to them. How do we
> >> verify scope in this model?
> > I think we need to find out how. One case is, the HW is designed to have
> > one single function, such as firmware update, then any image input
> > through firmware update API is within expectation, and the driver
> > should only serve the firmware update API. I think this is how the
> > N3000 is working now. If the HW is for another function, register itself
> > to serve another API, or another class driver.
> >
> > Another case is, the HW could do multiple types of tasks depending on
> > the content of the image, such as firmware update, image verification,
> > or assumably power off the card ... There should be some mechanism for
> > the driver to only accept the right image according to what API is called.
> > Or the user may input an image named update_the_card.img through
> > firmware update API and finally get the card off. Having some headers
> > readable by host for the operation type? Or, some HW interface for host
> > to apply the operation type as well as the image, let the HW verify?
> > Let's think about it.
> I'm not sure if I am following your thinking here. The HW, of course,
> verifies authentication of the image and acts according to the image
> type. I don't think it is reasonable to require the class driver to
> interpret the data to determine what it is. That implies that the
> class driver would have to know the header format and possible values.
> It also means that changes to the header format would require patches
> to the class driver.
> 
> The FPGA card is trusted by virtue of the fact that the customer
> purchased it and physically placed it in the machine. If the FPGA card
> (or the lower level driver) validates the image, then why does the
> Class driver need to be concerned about the file type? I think the
> purpose of the class driver is primarily to provide a common API and
> perform common functions so that they don't have to be replicated
> among similar low-level drivers. It is up to the low-level driver
> or the device itself to ensure that the data received is acceptable.
> 
> If the card receives data through the fpga-mgr upload facility that
> isn't strictly a firmware update, and if the lower-level driver or
> the card handles it and returns appropriate status - is that really
> a problem?
> >> As you have noted, most current cases result in a change to the
> >> card, and I suspect that it will remain that way. But that can't be
> >> guaranteed, and I'm not convinced that a host driver needs to be
> >> concerned about it.
> > A host driver should know what is done, in some abstraction level.
> > I think updating the persistent storage is an acceptable abstraction
> > in FPGA domain, so I'd like to extend it in FPGA manager. But doing
> > anything according to the image is not.
> By host driver, do you mean the class driver? Or the lower-level device
> driver?

The class driver.

> 
> It seems to me that you are saying that self-describing images are not
> acceptable? Or at least they are not acceptable payload for an FPGA
> Manager firmware-update API?

For N3000, we are working on the persistent storage update APIs, which is
a much simpler working model, no runtime device change, and needs no
device removal & re-enumeration.

But if you need to extend something more that would potentially changes
the behavior of the running devices on FPGA, device removal &
re-enumeration are needed so that the system knows what devices are
changed.

> 
> The FPGA Image Load Framework was designed with the concept of
> transferring data to a device without imposing a purpose on the data.
> The expectation is that the lower-level driver or the device will
> validate the data. Is there something fundamentally wrong with that

I think there is something wrong here. As I said before, persistent
storage updating has different software process from some runtime
updating, so the class driver should be aware of what the HW engine
is doing.

Thanks,
Yilun

> approach? And if not, why couldn't we incorporate a similar image_load
> API into the FPGA Manager?
> 
> - Russ
> 
> >
> > Thanks,
> > Yilun
> >
> >> - Russ
> >>
> >>> Thanks,
> >>> Yilun
> >>>
> >>>> Thanks,
> >>>> - Russ
> >>>>
> >>>>> Thanks,
> >>>>> Yilun
> >>>>>
> >>>>>>> while
> >>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> >>>>>>> to know the exact physical position to store, may not, depends on what the
> >>>>>>> HW engines are.
> >>>>>>>
> >>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
> >>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
> >>>>>>>> See this email for a recap:
> >>>>>>>>
> >>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> >>>>>>>>
> >>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
> >>>>>>>> approach probably fits better than the sysfs implementation. At the time
> >>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
> >>>>>>> I think at least developers don't have to go through 2 sets of software
> >>>>>>> stacks which are of the same concept. And adding some new features like
> >>>>>>> optionally threading or canceling data transfer are also good to FPGA
> >>>>>>> region update. And the nvmem update could also be benifit from exsiting
> >>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
> >>>>>>>
> >>>>>>> I try to explain myself according to each of your concern from that mail
> >>>>>>> thread:
> >>>>>>>
> >>>>>>> Purpose of the 2 updates
> >>>>>>> ========================
> >>>>>>>
> >>>>>>>   As I said before, I think they are both data transfer devices. FPGA
> >>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
> >>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
> >>>>>>>
> >>>>>>> Extra APIs that are unique to nvmem update
> >>>>>>> ==========================================
> >>>>>>>
> >>>>>>>   cdev APIs for nvmem update:
> >>>>>>>     Yes we need to expand the functionality so we need them.
> >>>>>>>
> >>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
> >>>>>>>   region:
> >>>>>>>     These are features in later patchsets which are not submitted, but we
> >>>>>>>     could talk about it in advance. I think this is actually a FPGA region
> >>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
> >>>>>>>     and re-enumeration, or a single command to image_load HW may result system
> >>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
> >>>>>>>     user data, maybe we add support for update from nvmem later.
> >>>>>>>
> >>>>>>>   fpga-mgr state extend:
> >>>>>>>     I think it could be extended, The current states are not perfect,
> >>>>>>>     adding something like IDLE or READY is just fine.
> >>>>>>>
> >>>>>>>   fpga-mgr status extend:
> >>>>>>>     Add general error definitions as needed. If there is some status
> >>>>>>>     that is quite vendor specific, expose it in low-level driver.
> >>>>>>>
> >>>>>>>   remaining-size:
> >>>>>>>     Nice to have for all.
> >>>>>>>
> >>>>>>> Threading the update
> >>>>>>> ====================
> >>>>>>>
> >>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
> >>>>>>>   reprogrammer?
> >>>>>>>
> >>>>>>> Cancelling the update
> >>>>>>> ====================
> >>>>>>>
> >>>>>>>   Also a good option for FPGA region update if HW engine supports.
> >>>>>> These are all good points.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> - Russ
> >>>>>>> Thanks,
> >>>>>>> Yilun
> >>>>>>>
> >>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
> >>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
> >>>>>>>>>>>> differences like threading or canceling the update, which are
> >>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
> >>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>>>>>>>>>
> >>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
> >>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
> >>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
> >>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>>>>>>>>>> another topic.
> >>>>>>>> I'll give this some more thought and see if I can come up with some RFC
> >>>>>>>> patches.
> >>>>>>>>
> >>>>>>>> - Russ
> >>>>>>>>>>>> More discussion is appreciated.
> >>>>>>>>>>> I also think fpga_mgr could be extended.
> >>>>>>>>>>>
> >>>>>>>>>>> In this patchset,
> >>>>>>>>>>>
> >>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>>>>>>>>>
> >>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>>>>>>>>>
> >>>>>>>>>>> new bit/flag was added to fpga_image_info
> >>>>>>>>>>>
> >>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
> >>>>>>>>>>> this patchset.
> >>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >>>>>>>>>> the 2 processes are almost the same.
> >>>>>>>>>>
> >>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
> >>>>>>>>>> flags.
> >>>>>>>>>>
> >>>>>>>>>> How do you think?
> >>>>>>>>> A single set is fine.
> >>>>>>>>>
> >>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >>>>>>>>>
> >>>>>>>>> Tom
> >>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Yilun
> >>>>>>>>>>
> >>>>>>>>>>> Any other driver would do similar.
> >>>>>>>>>>>
> >>>>>>>>>>> Is this close to what you are thinking ?
> >>>>>>>>>>>
> >>>>>>>>>>> Tom
> >>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Yilun
> >>>>>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/18/21 1:13 AM, Xu Yilun wrote:
> On Fri, Oct 15, 2021 at 10:34:23AM -0700, Russ Weight wrote:
>>
>> On 10/14/21 7:51 PM, Xu Yilun wrote:
>>> On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
>>>> On 10/13/21 6:49 PM, Xu Yilun wrote:
>>>>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
>>>>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
>>>>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
>>>>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
>>>>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
>>>>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
>>>>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
>>>>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>>>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
>>>>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
>>>>>>>>>>>>>>> target device to authenticate and disposition the file data.
>>>>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
>>>>>>>>>>>>>> may include it in FPGA manager framework.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>>>>>>>>>>>> I need to consider more seriously than before.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
>>>>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
>>>>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>>>>>>>>>>>> supports updating both FPGA region and nvmem.
>>>>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
>>>>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
>>>>>>>>> acting as the FPGA region admin responsible for gating, transfering and
>>>>>>>>> re-enumerating.
>>>>>>>>>
>>>>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
>>>>>>>>>
>>>>>>>>>> driver has no knowledge about what the data is or where/if the data will
>>>>>>>>>> be stored.
>>>>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
>>>>>>>> FYI: This is not strictly correct. In a coming product there is a
>>>>>>>> case where the data will be stored in RAM. Richard Gong was also
>>>>>>>> looking to use this driver to validate an image without programming
>>>>>>>> or storing it. The fpga-image-load driver has no expectation that
>>>>>>>> the data will be stored in nvmem, or even that it will be stored
>>>>>>>> at all.
>>>>>>> OK, we can discuss that use case then. But fundamentally a driver should
>>>>>>> be clear what it is doing.
>>>>>> The lower-level driver is, of course, clear what it is doing. And the
>>>>>> FPGA Image Load Framework simply provides a consistent API and manages
>>>>>> a potentially long-running data transfer in the context of a kernel
>>>>>> worker thread.
>>>>>>
>>>>>> It sounds like you are saying that that is not "clear enough" in the
>>>>>> context of the FPGA Manager?
>>>>>>
>>>>>> The files that are used with Intel PAC devices are self-describing. The
>>>>>> user-space tools, the class driver and the lower-level driver just pass
>>>>>> the data through to the card BMC without any knowledge of the content,
>>>>>> purpose or final destination of the data.
>>>>>>
>>>>>> The card BMC will receive signed data, validate it, and process it as a
>>>>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
>>>>> I category all these actions as firmware update fully or partially on
>>>>> persistent storage. The FPGA Manager don't have to know the meaning of
>>>>> every byte on flash, but it should be aware the firmware is updated and
>>>>> the card may acts differently with a new firmware. This is the common
>>>>> working model for most of the FPGA cards so that we implement it in FPGA
>>>>> manager class. 
>>>>>
>>>>>> the n6000, it could also be part of a multi-step process for programming
>>>>>> SDM keys and the data may not be stored permanently.
>>>>> This is new to me, but seems to be different from firmware update, so lets
>>>>> think about it again.
>>>>>
>>>>>>> You may try to extend the FPGA framework to
>>>>>>> support nvmem storage, or image validation, but cannot say we feed the
>>>>>>> data to any engine undefined by the framework.
>>>>>> I'm not sure what you mean by "feed the data to any engine undefined by the
>>>>>> framework". I think the "engine" is the lower level driver/device that invokes
>>>>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
>>>>>> The fpga_mgr cannot control what driver invokes it.
>>>>>>
>>>>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
>>>>>> data. Meaning that a self-describing file alone is not acceptable?
>>>>> The class driver should define a reasonable working model and APIs.
>>>>> Updating the FPGA backup storage is good to me. But receiving a mystery
>>>>> box and do whatever it requires is not.
>>>>>
>>>>> Self-describing file is OK, encryption is OK, but either the class
>>>>> driver itself, or with the help of the low level driver, should make
>>>>> sure it works within its scope.
>>>> In our secure update process, the card BMC firmware authenticates
>>>> the data using the root entry hashes and will either reject the
>>>> data or perform some function based on the contents. Neither the
>>>> user-space, the class driver, nor the lower level driver know
>>>> what the contents are. It _is_ a "mystery box" to them. How do we
>>>> verify scope in this model?
>>> I think we need to find out how. One case is, the HW is designed to have
>>> one single function, such as firmware update, then any image input
>>> through firmware update API is within expectation, and the driver
>>> should only serve the firmware update API. I think this is how the
>>> N3000 is working now. If the HW is for another function, register itself
>>> to serve another API, or another class driver.
>>>
>>> Another case is, the HW could do multiple types of tasks depending on
>>> the content of the image, such as firmware update, image verification,
>>> or assumably power off the card ... There should be some mechanism for
>>> the driver to only accept the right image according to what API is called.
>>> Or the user may input an image named update_the_card.img through
>>> firmware update API and finally get the card off. Having some headers
>>> readable by host for the operation type? Or, some HW interface for host
>>> to apply the operation type as well as the image, let the HW verify?
>>> Let's think about it.
>> I'm not sure if I am following your thinking here. The HW, of course,
>> verifies authentication of the image and acts according to the image
>> type. I don't think it is reasonable to require the class driver to
>> interpret the data to determine what it is. That implies that the
>> class driver would have to know the header format and possible values.
>> It also means that changes to the header format would require patches
>> to the class driver.
>>
>> The FPGA card is trusted by virtue of the fact that the customer
>> purchased it and physically placed it in the machine. If the FPGA card
>> (or the lower level driver) validates the image, then why does the
>> Class driver need to be concerned about the file type? I think the
>> purpose of the class driver is primarily to provide a common API and
>> perform common functions so that they don't have to be replicated
>> among similar low-level drivers. It is up to the low-level driver
>> or the device itself to ensure that the data received is acceptable.
>>
>> If the card receives data through the fpga-mgr upload facility that
>> isn't strictly a firmware update, and if the lower-level driver or
>> the card handles it and returns appropriate status - is that really
>> a problem?
>>>> As you have noted, most current cases result in a change to the
>>>> card, and I suspect that it will remain that way. But that can't be
>>>> guaranteed, and I'm not convinced that a host driver needs to be
>>>> concerned about it.
>>> A host driver should know what is done, in some abstraction level.
>>> I think updating the persistent storage is an acceptable abstraction
>>> in FPGA domain, so I'd like to extend it in FPGA manager. But doing
>>> anything according to the image is not.
>> By host driver, do you mean the class driver? Or the lower-level device
>> driver?
> The class driver.
>
>> It seems to me that you are saying that self-describing images are not
>> acceptable? Or at least they are not acceptable payload for an FPGA
>> Manager firmware-update API?
> For N3000, we are working on the persistent storage update APIs, which is
> a much simpler working model, no runtime device change, and needs no
> device removal & re-enumeration.
>
> But if you need to extend something more that would potentially changes
> the behavior of the running devices on FPGA, device removal &
> re-enumeration are needed so that the system knows what devices are
> changed.
>
>> The FPGA Image Load Framework was designed with the concept of
>> transferring data to a device without imposing a purpose on the data.
>> The expectation is that the lower-level driver or the device will
>> validate the data. Is there something fundamentally wrong with that
> I think there is something wrong here. As I said before, persistent
> storage updating has different software process from some runtime
> updating, so the class driver should be aware of what the HW engine
> is doing.
So far, there are no self-describing images that cause a
change in run-time behavior, and I don't think that will
happen for the very reason that the class-driver would
need to know about it.

When I asserted that not all self-describing images are
changing firmware, I did not mean to imply that they change
run-time behavior; they do not. They are part of a multi-
step update of firmware. However, looking at each part of
the sequence independently, there is at least one instance
where a certificate is stored in RAM for temporary use.
When the driver exits from this call, there is no firmware
change. There is also no change in run-time behavior.

I'm thinking we could have different IOCTLs:

(1) firmware  update (address, size, purpose provided
    with the image)
(2) image upload (self-describing files)
(3) image validation

These would all use most of the same code, but the FPGA
Manager flags and structure fields would be set differently.

- Russ
>
> Thanks,
> Yilun
>
>> approach? And if not, why couldn't we incorporate a similar image_load
>> API into the FPGA Manager?
>>
>> - Russ
>>
>>> Thanks,
>>> Yilun
>>>
>>>> - Russ
>>>>
>>>>> Thanks,
>>>>> Yilun
>>>>>
>>>>>> Thanks,
>>>>>> - Russ
>>>>>>
>>>>>>> Thanks,
>>>>>>> Yilun
>>>>>>>
>>>>>>>>> while
>>>>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
>>>>>>>>> to know the exact physical position to store, may not, depends on what the
>>>>>>>>> HW engines are.
>>>>>>>>>
>>>>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
>>>>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
>>>>>>>>>> See this email for a recap:
>>>>>>>>>>
>>>>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
>>>>>>>>>>
>>>>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
>>>>>>>>>> approach probably fits better than the sysfs implementation. At the time
>>>>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
>>>>>>>>> I think at least developers don't have to go through 2 sets of software
>>>>>>>>> stacks which are of the same concept. And adding some new features like
>>>>>>>>> optionally threading or canceling data transfer are also good to FPGA
>>>>>>>>> region update. And the nvmem update could also be benifit from exsiting
>>>>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
>>>>>>>>>
>>>>>>>>> I try to explain myself according to each of your concern from that mail
>>>>>>>>> thread:
>>>>>>>>>
>>>>>>>>> Purpose of the 2 updates
>>>>>>>>> ========================
>>>>>>>>>
>>>>>>>>>   As I said before, I think they are both data transfer devices. FPGA
>>>>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
>>>>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
>>>>>>>>>
>>>>>>>>> Extra APIs that are unique to nvmem update
>>>>>>>>> ==========================================
>>>>>>>>>
>>>>>>>>>   cdev APIs for nvmem update:
>>>>>>>>>     Yes we need to expand the functionality so we need them.
>>>>>>>>>
>>>>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
>>>>>>>>>   region:
>>>>>>>>>     These are features in later patchsets which are not submitted, but we
>>>>>>>>>     could talk about it in advance. I think this is actually a FPGA region
>>>>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
>>>>>>>>>     and re-enumeration, or a single command to image_load HW may result system
>>>>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
>>>>>>>>>     user data, maybe we add support for update from nvmem later.
>>>>>>>>>
>>>>>>>>>   fpga-mgr state extend:
>>>>>>>>>     I think it could be extended, The current states are not perfect,
>>>>>>>>>     adding something like IDLE or READY is just fine.
>>>>>>>>>
>>>>>>>>>   fpga-mgr status extend:
>>>>>>>>>     Add general error definitions as needed. If there is some status
>>>>>>>>>     that is quite vendor specific, expose it in low-level driver.
>>>>>>>>>
>>>>>>>>>   remaining-size:
>>>>>>>>>     Nice to have for all.
>>>>>>>>>
>>>>>>>>> Threading the update
>>>>>>>>> ====================
>>>>>>>>>
>>>>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>>>>>>>>>   reprogrammer?
>>>>>>>>>
>>>>>>>>> Cancelling the update
>>>>>>>>> ====================
>>>>>>>>>
>>>>>>>>>   Also a good option for FPGA region update if HW engine supports.
>>>>>>>> These are all good points.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> - Russ
>>>>>>>>> Thanks,
>>>>>>>>> Yilun
>>>>>>>>>
>>>>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
>>>>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
>>>>>>>>>>>>>> differences like threading or canceling the update, which are
>>>>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
>>>>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
>>>>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
>>>>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
>>>>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>>>>>>>>>>>> another topic.
>>>>>>>>>> I'll give this some more thought and see if I can come up with some RFC
>>>>>>>>>> patches.
>>>>>>>>>>
>>>>>>>>>> - Russ
>>>>>>>>>>>>>> More discussion is appreciated.
>>>>>>>>>>>>> I also think fpga_mgr could be extended.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In this patchset,
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>>>>>>>>>>>
>>>>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>>>>>>>>>>>
>>>>>>>>>>>>> new bit/flag was added to fpga_image_info
>>>>>>>>>>>>>
>>>>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
>>>>>>>>>>>>> this patchset.
>>>>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>>>>>>>>>>>> the 2 processes are almost the same.
>>>>>>>>>>>>
>>>>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>>>>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
>>>>>>>>>>>> flags.
>>>>>>>>>>>>
>>>>>>>>>>>> How do you think?
>>>>>>>>>>> A single set is fine.
>>>>>>>>>>>
>>>>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>>>>>>>>>>>
>>>>>>>>>>> Tom
>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Yilun
>>>>>>>>>>>>
>>>>>>>>>>>>> Any other driver would do similar.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is this close to what you are thinking ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Yilun
>>>>>>>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Mon, Oct 18, 2021 at 09:24:08AM -0700, Russ Weight wrote:
> 
> 
> On 10/18/21 1:13 AM, Xu Yilun wrote:
> > On Fri, Oct 15, 2021 at 10:34:23AM -0700, Russ Weight wrote:
> >>
> >> On 10/14/21 7:51 PM, Xu Yilun wrote:
> >>> On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
> >>>> On 10/13/21 6:49 PM, Xu Yilun wrote:
> >>>>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
> >>>>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
> >>>>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
> >>>>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
> >>>>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> >>>>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
> >>>>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
> >>>>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>>>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
> >>>>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
> >>>>>>>>>>>>>>> target device to authenticate and disposition the file data.
> >>>>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
> >>>>>>>>>>>>>> may include it in FPGA manager framework.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>>>>>>>>>>>> I need to consider more seriously than before.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
> >>>>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
> >>>>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>>>>>>>>>>>> supports updating both FPGA region and nvmem.
> >>>>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
> >>>>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> >>>>>>>>> acting as the FPGA region admin responsible for gating, transfering and
> >>>>>>>>> re-enumerating.
> >>>>>>>>>
> >>>>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
> >>>>>>>>>
> >>>>>>>>>> driver has no knowledge about what the data is or where/if the data will
> >>>>>>>>>> be stored.
> >>>>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
> >>>>>>>> FYI: This is not strictly correct. In a coming product there is a
> >>>>>>>> case where the data will be stored in RAM. Richard Gong was also
> >>>>>>>> looking to use this driver to validate an image without programming
> >>>>>>>> or storing it. The fpga-image-load driver has no expectation that
> >>>>>>>> the data will be stored in nvmem, or even that it will be stored
> >>>>>>>> at all.
> >>>>>>> OK, we can discuss that use case then. But fundamentally a driver should
> >>>>>>> be clear what it is doing.
> >>>>>> The lower-level driver is, of course, clear what it is doing. And the
> >>>>>> FPGA Image Load Framework simply provides a consistent API and manages
> >>>>>> a potentially long-running data transfer in the context of a kernel
> >>>>>> worker thread.
> >>>>>>
> >>>>>> It sounds like you are saying that that is not "clear enough" in the
> >>>>>> context of the FPGA Manager?
> >>>>>>
> >>>>>> The files that are used with Intel PAC devices are self-describing. The
> >>>>>> user-space tools, the class driver and the lower-level driver just pass
> >>>>>> the data through to the card BMC without any knowledge of the content,
> >>>>>> purpose or final destination of the data.
> >>>>>>
> >>>>>> The card BMC will receive signed data, validate it, and process it as a
> >>>>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
> >>>>> I category all these actions as firmware update fully or partially on
> >>>>> persistent storage. The FPGA Manager don't have to know the meaning of
> >>>>> every byte on flash, but it should be aware the firmware is updated and
> >>>>> the card may acts differently with a new firmware. This is the common
> >>>>> working model for most of the FPGA cards so that we implement it in FPGA
> >>>>> manager class. 
> >>>>>
> >>>>>> the n6000, it could also be part of a multi-step process for programming
> >>>>>> SDM keys and the data may not be stored permanently.
> >>>>> This is new to me, but seems to be different from firmware update, so lets
> >>>>> think about it again.
> >>>>>
> >>>>>>> You may try to extend the FPGA framework to
> >>>>>>> support nvmem storage, or image validation, but cannot say we feed the
> >>>>>>> data to any engine undefined by the framework.
> >>>>>> I'm not sure what you mean by "feed the data to any engine undefined by the
> >>>>>> framework". I think the "engine" is the lower level driver/device that invokes
> >>>>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
> >>>>>> The fpga_mgr cannot control what driver invokes it.
> >>>>>>
> >>>>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
> >>>>>> data. Meaning that a self-describing file alone is not acceptable?
> >>>>> The class driver should define a reasonable working model and APIs.
> >>>>> Updating the FPGA backup storage is good to me. But receiving a mystery
> >>>>> box and do whatever it requires is not.
> >>>>>
> >>>>> Self-describing file is OK, encryption is OK, but either the class
> >>>>> driver itself, or with the help of the low level driver, should make
> >>>>> sure it works within its scope.
> >>>> In our secure update process, the card BMC firmware authenticates
> >>>> the data using the root entry hashes and will either reject the
> >>>> data or perform some function based on the contents. Neither the
> >>>> user-space, the class driver, nor the lower level driver know
> >>>> what the contents are. It _is_ a "mystery box" to them. How do we
> >>>> verify scope in this model?
> >>> I think we need to find out how. One case is, the HW is designed to have
> >>> one single function, such as firmware update, then any image input
> >>> through firmware update API is within expectation, and the driver
> >>> should only serve the firmware update API. I think this is how the
> >>> N3000 is working now. If the HW is for another function, register itself
> >>> to serve another API, or another class driver.
> >>>
> >>> Another case is, the HW could do multiple types of tasks depending on
> >>> the content of the image, such as firmware update, image verification,
> >>> or assumably power off the card ... There should be some mechanism for
> >>> the driver to only accept the right image according to what API is called.
> >>> Or the user may input an image named update_the_card.img through
> >>> firmware update API and finally get the card off. Having some headers
> >>> readable by host for the operation type? Or, some HW interface for host
> >>> to apply the operation type as well as the image, let the HW verify?
> >>> Let's think about it.
> >> I'm not sure if I am following your thinking here. The HW, of course,
> >> verifies authentication of the image and acts according to the image
> >> type. I don't think it is reasonable to require the class driver to
> >> interpret the data to determine what it is. That implies that the
> >> class driver would have to know the header format and possible values.
> >> It also means that changes to the header format would require patches
> >> to the class driver.
> >>
> >> The FPGA card is trusted by virtue of the fact that the customer
> >> purchased it and physically placed it in the machine. If the FPGA card
> >> (or the lower level driver) validates the image, then why does the
> >> Class driver need to be concerned about the file type? I think the
> >> purpose of the class driver is primarily to provide a common API and
> >> perform common functions so that they don't have to be replicated
> >> among similar low-level drivers. It is up to the low-level driver
> >> or the device itself to ensure that the data received is acceptable.
> >>
> >> If the card receives data through the fpga-mgr upload facility that
> >> isn't strictly a firmware update, and if the lower-level driver or
> >> the card handles it and returns appropriate status - is that really
> >> a problem?
> >>>> As you have noted, most current cases result in a change to the
> >>>> card, and I suspect that it will remain that way. But that can't be
> >>>> guaranteed, and I'm not convinced that a host driver needs to be
> >>>> concerned about it.
> >>> A host driver should know what is done, in some abstraction level.
> >>> I think updating the persistent storage is an acceptable abstraction
> >>> in FPGA domain, so I'd like to extend it in FPGA manager. But doing
> >>> anything according to the image is not.
> >> By host driver, do you mean the class driver? Or the lower-level device
> >> driver?
> > The class driver.
> >
> >> It seems to me that you are saying that self-describing images are not
> >> acceptable? Or at least they are not acceptable payload for an FPGA
> >> Manager firmware-update API?
> > For N3000, we are working on the persistent storage update APIs, which is
> > a much simpler working model, no runtime device change, and needs no
> > device removal & re-enumeration.
> >
> > But if you need to extend something more that would potentially changes
> > the behavior of the running devices on FPGA, device removal &
> > re-enumeration are needed so that the system knows what devices are
> > changed.
> >
> >> The FPGA Image Load Framework was designed with the concept of
> >> transferring data to a device without imposing a purpose on the data.
> >> The expectation is that the lower-level driver or the device will
> >> validate the data. Is there something fundamentally wrong with that
> > I think there is something wrong here. As I said before, persistent
> > storage updating has different software process from some runtime
> > updating, so the class driver should be aware of what the HW engine
> > is doing.
> So far, there are no self-describing images that cause a
> change in run-time behavior, and I don't think that will
> happen for the very reason that the class-driver would
> need to know about it.

Again, the class driver needs to know what is happening, at some
abstraction level, to ensure the system is aligned with the HW state.

If the class driver cannot tell the detail, it has to assume the
whole FPGA region will be changed, and removal & re-enumeration is
needed.

> 
> When I asserted that not all self-describing images are
> changing firmware, I did not mean to imply that they change
> run-time behavior; they do not. They are part of a multi-
> step update of firmware. However, looking at each part of
> the sequence independently, there is at least one instance
> where a certificate is stored in RAM for temporary use.
> When the driver exits from this call, there is no firmware
> change. There is also no change in run-time behavior.
> 
> I'm thinking we could have different IOCTLs:
> 
> (1) firmware  update (address, size, purpose provided
>     with the image)

Will the firmware update use the self-describing files?

> (2) image upload (self-describing files)

If both 1 & 2 use self-describing files, how the class driver verifies
the type of the file without looking into the file?

For example, if a user calls a firmware update API but inputs an image
upload file, will the class driver block the call? How?

> (3) image validation
> 
> These would all use most of the same code, but the FPGA
> Manager flags and structure fields would be set differently.

This is good to me.

Thanks,
Yilun

> 
> - Russ
> >
> > Thanks,
> > Yilun
> >
> >> approach? And if not, why couldn't we incorporate a similar image_load
> >> API into the FPGA Manager?
> >>
> >> - Russ
> >>
> >>> Thanks,
> >>> Yilun
> >>>
> >>>> - Russ
> >>>>
> >>>>> Thanks,
> >>>>> Yilun
> >>>>>
> >>>>>> Thanks,
> >>>>>> - Russ
> >>>>>>
> >>>>>>> Thanks,
> >>>>>>> Yilun
> >>>>>>>
> >>>>>>>>> while
> >>>>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> >>>>>>>>> to know the exact physical position to store, may not, depends on what the
> >>>>>>>>> HW engines are.
> >>>>>>>>>
> >>>>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
> >>>>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
> >>>>>>>>>> See this email for a recap:
> >>>>>>>>>>
> >>>>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> >>>>>>>>>>
> >>>>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
> >>>>>>>>>> approach probably fits better than the sysfs implementation. At the time
> >>>>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
> >>>>>>>>> I think at least developers don't have to go through 2 sets of software
> >>>>>>>>> stacks which are of the same concept. And adding some new features like
> >>>>>>>>> optionally threading or canceling data transfer are also good to FPGA
> >>>>>>>>> region update. And the nvmem update could also be benifit from exsiting
> >>>>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
> >>>>>>>>>
> >>>>>>>>> I try to explain myself according to each of your concern from that mail
> >>>>>>>>> thread:
> >>>>>>>>>
> >>>>>>>>> Purpose of the 2 updates
> >>>>>>>>> ========================
> >>>>>>>>>
> >>>>>>>>>   As I said before, I think they are both data transfer devices. FPGA
> >>>>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
> >>>>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
> >>>>>>>>>
> >>>>>>>>> Extra APIs that are unique to nvmem update
> >>>>>>>>> ==========================================
> >>>>>>>>>
> >>>>>>>>>   cdev APIs for nvmem update:
> >>>>>>>>>     Yes we need to expand the functionality so we need them.
> >>>>>>>>>
> >>>>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
> >>>>>>>>>   region:
> >>>>>>>>>     These are features in later patchsets which are not submitted, but we
> >>>>>>>>>     could talk about it in advance. I think this is actually a FPGA region
> >>>>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
> >>>>>>>>>     and re-enumeration, or a single command to image_load HW may result system
> >>>>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
> >>>>>>>>>     user data, maybe we add support for update from nvmem later.
> >>>>>>>>>
> >>>>>>>>>   fpga-mgr state extend:
> >>>>>>>>>     I think it could be extended, The current states are not perfect,
> >>>>>>>>>     adding something like IDLE or READY is just fine.
> >>>>>>>>>
> >>>>>>>>>   fpga-mgr status extend:
> >>>>>>>>>     Add general error definitions as needed. If there is some status
> >>>>>>>>>     that is quite vendor specific, expose it in low-level driver.
> >>>>>>>>>
> >>>>>>>>>   remaining-size:
> >>>>>>>>>     Nice to have for all.
> >>>>>>>>>
> >>>>>>>>> Threading the update
> >>>>>>>>> ====================
> >>>>>>>>>
> >>>>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
> >>>>>>>>>   reprogrammer?
> >>>>>>>>>
> >>>>>>>>> Cancelling the update
> >>>>>>>>> ====================
> >>>>>>>>>
> >>>>>>>>>   Also a good option for FPGA region update if HW engine supports.
> >>>>>>>> These are all good points.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> - Russ
> >>>>>>>>> Thanks,
> >>>>>>>>> Yilun
> >>>>>>>>>
> >>>>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
> >>>>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
> >>>>>>>>>>>>>> differences like threading or canceling the update, which are
> >>>>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
> >>>>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
> >>>>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
> >>>>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
> >>>>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>>>>>>>>>>>> another topic.
> >>>>>>>>>> I'll give this some more thought and see if I can come up with some RFC
> >>>>>>>>>> patches.
> >>>>>>>>>>
> >>>>>>>>>> - Russ
> >>>>>>>>>>>>>> More discussion is appreciated.
> >>>>>>>>>>>>> I also think fpga_mgr could be extended.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In this patchset,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> new bit/flag was added to fpga_image_info
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
> >>>>>>>>>>>>> this patchset.
> >>>>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >>>>>>>>>>>> the 2 processes are almost the same.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >>>>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
> >>>>>>>>>>>> flags.
> >>>>>>>>>>>>
> >>>>>>>>>>>> How do you think?
> >>>>>>>>>>> A single set is fine.
> >>>>>>>>>>>
> >>>>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >>>>>>>>>>>
> >>>>>>>>>>> Tom
> >>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Yilun
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Any other driver would do similar.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Is this close to what you are thinking ?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Tom
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Yilun
> >>>>>>>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/18/21 7:53 PM, Xu Yilun wrote:
> On Mon, Oct 18, 2021 at 09:24:08AM -0700, Russ Weight wrote:
>>
>> On 10/18/21 1:13 AM, Xu Yilun wrote:
>>> On Fri, Oct 15, 2021 at 10:34:23AM -0700, Russ Weight wrote:
>>>> On 10/14/21 7:51 PM, Xu Yilun wrote:
>>>>> On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
>>>>>> On 10/13/21 6:49 PM, Xu Yilun wrote:
>>>>>>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
>>>>>>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
>>>>>>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
>>>>>>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
>>>>>>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
>>>>>>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
>>>>>>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
>>>>>>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>>>>>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>>>>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>>>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
>>>>>>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>>>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>>>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
>>>>>>>>>>>>>>>>> target device to authenticate and disposition the file data.
>>>>>>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
>>>>>>>>>>>>>>>> may include it in FPGA manager framework.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>>>>>>>>>>>>>> I need to consider more seriously than before.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
>>>>>>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>>>>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
>>>>>>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>>>>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>>>>>>>>>>>>>> supports updating both FPGA region and nvmem.
>>>>>>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
>>>>>>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
>>>>>>>>>>> acting as the FPGA region admin responsible for gating, transfering and
>>>>>>>>>>> re-enumerating.
>>>>>>>>>>>
>>>>>>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
>>>>>>>>>>>
>>>>>>>>>>>> driver has no knowledge about what the data is or where/if the data will
>>>>>>>>>>>> be stored.
>>>>>>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
>>>>>>>>>> FYI: This is not strictly correct. In a coming product there is a
>>>>>>>>>> case where the data will be stored in RAM. Richard Gong was also
>>>>>>>>>> looking to use this driver to validate an image without programming
>>>>>>>>>> or storing it. The fpga-image-load driver has no expectation that
>>>>>>>>>> the data will be stored in nvmem, or even that it will be stored
>>>>>>>>>> at all.
>>>>>>>>> OK, we can discuss that use case then. But fundamentally a driver should
>>>>>>>>> be clear what it is doing.
>>>>>>>> The lower-level driver is, of course, clear what it is doing. And the
>>>>>>>> FPGA Image Load Framework simply provides a consistent API and manages
>>>>>>>> a potentially long-running data transfer in the context of a kernel
>>>>>>>> worker thread.
>>>>>>>>
>>>>>>>> It sounds like you are saying that that is not "clear enough" in the
>>>>>>>> context of the FPGA Manager?
>>>>>>>>
>>>>>>>> The files that are used with Intel PAC devices are self-describing. The
>>>>>>>> user-space tools, the class driver and the lower-level driver just pass
>>>>>>>> the data through to the card BMC without any knowledge of the content,
>>>>>>>> purpose or final destination of the data.
>>>>>>>>
>>>>>>>> The card BMC will receive signed data, validate it, and process it as a
>>>>>>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
>>>>>>> I category all these actions as firmware update fully or partially on
>>>>>>> persistent storage. The FPGA Manager don't have to know the meaning of
>>>>>>> every byte on flash, but it should be aware the firmware is updated and
>>>>>>> the card may acts differently with a new firmware. This is the common
>>>>>>> working model for most of the FPGA cards so that we implement it in FPGA
>>>>>>> manager class. 
>>>>>>>
>>>>>>>> the n6000, it could also be part of a multi-step process for programming
>>>>>>>> SDM keys and the data may not be stored permanently.
>>>>>>> This is new to me, but seems to be different from firmware update, so lets
>>>>>>> think about it again.
>>>>>>>
>>>>>>>>> You may try to extend the FPGA framework to
>>>>>>>>> support nvmem storage, or image validation, but cannot say we feed the
>>>>>>>>> data to any engine undefined by the framework.
>>>>>>>> I'm not sure what you mean by "feed the data to any engine undefined by the
>>>>>>>> framework". I think the "engine" is the lower level driver/device that invokes
>>>>>>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
>>>>>>>> The fpga_mgr cannot control what driver invokes it.
>>>>>>>>
>>>>>>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
>>>>>>>> data. Meaning that a self-describing file alone is not acceptable?
>>>>>>> The class driver should define a reasonable working model and APIs.
>>>>>>> Updating the FPGA backup storage is good to me. But receiving a mystery
>>>>>>> box and do whatever it requires is not.
>>>>>>>
>>>>>>> Self-describing file is OK, encryption is OK, but either the class
>>>>>>> driver itself, or with the help of the low level driver, should make
>>>>>>> sure it works within its scope.
>>>>>> In our secure update process, the card BMC firmware authenticates
>>>>>> the data using the root entry hashes and will either reject the
>>>>>> data or perform some function based on the contents. Neither the
>>>>>> user-space, the class driver, nor the lower level driver know
>>>>>> what the contents are. It _is_ a "mystery box" to them. How do we
>>>>>> verify scope in this model?
>>>>> I think we need to find out how. One case is, the HW is designed to have
>>>>> one single function, such as firmware update, then any image input
>>>>> through firmware update API is within expectation, and the driver
>>>>> should only serve the firmware update API. I think this is how the
>>>>> N3000 is working now. If the HW is for another function, register itself
>>>>> to serve another API, or another class driver.
>>>>>
>>>>> Another case is, the HW could do multiple types of tasks depending on
>>>>> the content of the image, such as firmware update, image verification,
>>>>> or assumably power off the card ... There should be some mechanism for
>>>>> the driver to only accept the right image according to what API is called.
>>>>> Or the user may input an image named update_the_card.img through
>>>>> firmware update API and finally get the card off. Having some headers
>>>>> readable by host for the operation type? Or, some HW interface for host
>>>>> to apply the operation type as well as the image, let the HW verify?
>>>>> Let's think about it.
>>>> I'm not sure if I am following your thinking here. The HW, of course,
>>>> verifies authentication of the image and acts according to the image
>>>> type. I don't think it is reasonable to require the class driver to
>>>> interpret the data to determine what it is. That implies that the
>>>> class driver would have to know the header format and possible values.
>>>> It also means that changes to the header format would require patches
>>>> to the class driver.
>>>>
>>>> The FPGA card is trusted by virtue of the fact that the customer
>>>> purchased it and physically placed it in the machine. If the FPGA card
>>>> (or the lower level driver) validates the image, then why does the
>>>> Class driver need to be concerned about the file type? I think the
>>>> purpose of the class driver is primarily to provide a common API and
>>>> perform common functions so that they don't have to be replicated
>>>> among similar low-level drivers. It is up to the low-level driver
>>>> or the device itself to ensure that the data received is acceptable.
>>>>
>>>> If the card receives data through the fpga-mgr upload facility that
>>>> isn't strictly a firmware update, and if the lower-level driver or
>>>> the card handles it and returns appropriate status - is that really
>>>> a problem?
>>>>>> As you have noted, most current cases result in a change to the
>>>>>> card, and I suspect that it will remain that way. But that can't be
>>>>>> guaranteed, and I'm not convinced that a host driver needs to be
>>>>>> concerned about it.
>>>>> A host driver should know what is done, in some abstraction level.
>>>>> I think updating the persistent storage is an acceptable abstraction
>>>>> in FPGA domain, so I'd like to extend it in FPGA manager. But doing
>>>>> anything according to the image is not.
>>>> By host driver, do you mean the class driver? Or the lower-level device
>>>> driver?
>>> The class driver.
>>>
>>>> It seems to me that you are saying that self-describing images are not
>>>> acceptable? Or at least they are not acceptable payload for an FPGA
>>>> Manager firmware-update API?
>>> For N3000, we are working on the persistent storage update APIs, which is
>>> a much simpler working model, no runtime device change, and needs no
>>> device removal & re-enumeration.
>>>
>>> But if you need to extend something more that would potentially changes
>>> the behavior of the running devices on FPGA, device removal &
>>> re-enumeration are needed so that the system knows what devices are
>>> changed.
>>>
>>>> The FPGA Image Load Framework was designed with the concept of
>>>> transferring data to a device without imposing a purpose on the data.
>>>> The expectation is that the lower-level driver or the device will
>>>> validate the data. Is there something fundamentally wrong with that
>>> I think there is something wrong here. As I said before, persistent
>>> storage updating has different software process from some runtime
>>> updating, so the class driver should be aware of what the HW engine
>>> is doing.
>> So far, there are no self-describing images that cause a
>> change in run-time behavior, and I don't think that will
>> happen for the very reason that the class-driver would
>> need to know about it.
> Again, the class driver needs to know what is happening, at some
> abstraction level, to ensure the system is aligned with the HW state.
>
> If the class driver cannot tell the detail, it has to assume the
> whole FPGA region will be changed, and removal & re-enumeration is
> needed.
So we make it a requirement that the self-describing files
cannot make changes that require the class driver to manage
state.
>
>> When I asserted that not all self-describing images are
>> changing firmware, I did not mean to imply that they change
>> run-time behavior; they do not. They are part of a multi-
>> step update of firmware. However, looking at each part of
>> the sequence independently, there is at least one instance
>> where a certificate is stored in RAM for temporary use.
>> When the driver exits from this call, there is no firmware
>> change. There is also no change in run-time behavior.
>>
>> I'm thinking we could have different IOCTLs:
>>
>> (1) firmware  update (address, size, purpose provided
>>     with the image)
> Will the firmware update use the self-describing files?
The firmware update option would be for files that
are not self-describing.
>
>> (2) image upload (self-describing files)
> If both 1 & 2 use self-describing files, how the class driver verifies
> the type of the file without looking into the file?
Only 2 would use self-describing files.

- Russ
>
> For example, if a user calls a firmware update API but inputs an image
> upload file, will the class driver block the call? How?
>
>> (3) image validation
>>
>> These would all use most of the same code, but the FPGA
>> Manager flags and structure fields would be set differently.
> This is good to me.
>
> Thanks,
> Yilun
>
>> - Russ
>>> Thanks,
>>> Yilun
>>>
>>>> approach? And if not, why couldn't we incorporate a similar image_load
>>>> API into the FPGA Manager?
>>>>
>>>> - Russ
>>>>
>>>>> Thanks,
>>>>> Yilun
>>>>>
>>>>>> - Russ
>>>>>>
>>>>>>> Thanks,
>>>>>>> Yilun
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> - Russ
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Yilun
>>>>>>>>>
>>>>>>>>>>> while
>>>>>>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
>>>>>>>>>>> to know the exact physical position to store, may not, depends on what the
>>>>>>>>>>> HW engines are.
>>>>>>>>>>>
>>>>>>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
>>>>>>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
>>>>>>>>>>>> See this email for a recap:
>>>>>>>>>>>>
>>>>>>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
>>>>>>>>>>>>
>>>>>>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
>>>>>>>>>>>> approach probably fits better than the sysfs implementation. At the time
>>>>>>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
>>>>>>>>>>> I think at least developers don't have to go through 2 sets of software
>>>>>>>>>>> stacks which are of the same concept. And adding some new features like
>>>>>>>>>>> optionally threading or canceling data transfer are also good to FPGA
>>>>>>>>>>> region update. And the nvmem update could also be benifit from exsiting
>>>>>>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
>>>>>>>>>>>
>>>>>>>>>>> I try to explain myself according to each of your concern from that mail
>>>>>>>>>>> thread:
>>>>>>>>>>>
>>>>>>>>>>> Purpose of the 2 updates
>>>>>>>>>>> ========================
>>>>>>>>>>>
>>>>>>>>>>>   As I said before, I think they are both data transfer devices. FPGA
>>>>>>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
>>>>>>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
>>>>>>>>>>>
>>>>>>>>>>> Extra APIs that are unique to nvmem update
>>>>>>>>>>> ==========================================
>>>>>>>>>>>
>>>>>>>>>>>   cdev APIs for nvmem update:
>>>>>>>>>>>     Yes we need to expand the functionality so we need them.
>>>>>>>>>>>
>>>>>>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
>>>>>>>>>>>   region:
>>>>>>>>>>>     These are features in later patchsets which are not submitted, but we
>>>>>>>>>>>     could talk about it in advance. I think this is actually a FPGA region
>>>>>>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
>>>>>>>>>>>     and re-enumeration, or a single command to image_load HW may result system
>>>>>>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
>>>>>>>>>>>     user data, maybe we add support for update from nvmem later.
>>>>>>>>>>>
>>>>>>>>>>>   fpga-mgr state extend:
>>>>>>>>>>>     I think it could be extended, The current states are not perfect,
>>>>>>>>>>>     adding something like IDLE or READY is just fine.
>>>>>>>>>>>
>>>>>>>>>>>   fpga-mgr status extend:
>>>>>>>>>>>     Add general error definitions as needed. If there is some status
>>>>>>>>>>>     that is quite vendor specific, expose it in low-level driver.
>>>>>>>>>>>
>>>>>>>>>>>   remaining-size:
>>>>>>>>>>>     Nice to have for all.
>>>>>>>>>>>
>>>>>>>>>>> Threading the update
>>>>>>>>>>> ====================
>>>>>>>>>>>
>>>>>>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>>>>>>>>>>>   reprogrammer?
>>>>>>>>>>>
>>>>>>>>>>> Cancelling the update
>>>>>>>>>>> ====================
>>>>>>>>>>>
>>>>>>>>>>>   Also a good option for FPGA region update if HW engine supports.
>>>>>>>>>> These are all good points.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> - Russ
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Yilun
>>>>>>>>>>>
>>>>>>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
>>>>>>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>>>>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
>>>>>>>>>>>>>>>> differences like threading or canceling the update, which are
>>>>>>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>>>>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
>>>>>>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>>>>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>>>>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
>>>>>>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
>>>>>>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>>>>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>>>>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
>>>>>>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>>>>>>>>>>>>>> another topic.
>>>>>>>>>>>> I'll give this some more thought and see if I can come up with some RFC
>>>>>>>>>>>> patches.
>>>>>>>>>>>>
>>>>>>>>>>>> - Russ
>>>>>>>>>>>>>>>> More discussion is appreciated.
>>>>>>>>>>>>>>> I also think fpga_mgr could be extended.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In this patchset,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> new bit/flag was added to fpga_image_info
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
>>>>>>>>>>>>>>> this patchset.
>>>>>>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>>>>>>>>>>>>>> the 2 processes are almost the same.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>>>>>>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
>>>>>>>>>>>>>> flags.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How do you think?
>>>>>>>>>>>>> A single set is fine.
>>>>>>>>>>>>>
>>>>>>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Yilun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any other driver would do similar.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is this close to what you are thinking ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Yilun
>>>>>>>>>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Tue, Oct 19, 2021 at 08:09:46AM -0700, Russ Weight wrote:
> 
> 
> On 10/18/21 7:53 PM, Xu Yilun wrote:
> > On Mon, Oct 18, 2021 at 09:24:08AM -0700, Russ Weight wrote:
> >>
> >> On 10/18/21 1:13 AM, Xu Yilun wrote:
> >>> On Fri, Oct 15, 2021 at 10:34:23AM -0700, Russ Weight wrote:
> >>>> On 10/14/21 7:51 PM, Xu Yilun wrote:
> >>>>> On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
> >>>>>> On 10/13/21 6:49 PM, Xu Yilun wrote:
> >>>>>>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
> >>>>>>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
> >>>>>>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
> >>>>>>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
> >>>>>>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
> >>>>>>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
> >>>>>>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
> >>>>>>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
> >>>>>>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
> >>>>>>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
> >>>>>>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
> >>>>>>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
> >>>>>>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
> >>>>>>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
> >>>>>>>>>>>>>>>>> target device to authenticate and disposition the file data.
> >>>>>>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
> >>>>>>>>>>>>>>>> may include it in FPGA manager framework.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
> >>>>>>>>>>>>>>>> I need to consider more seriously than before.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
> >>>>>>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
> >>>>>>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
> >>>>>>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
> >>>>>>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
> >>>>>>>>>>>>>>>> supports updating both FPGA region and nvmem.
> >>>>>>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
> >>>>>>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
> >>>>>>>>>>> acting as the FPGA region admin responsible for gating, transfering and
> >>>>>>>>>>> re-enumerating.
> >>>>>>>>>>>
> >>>>>>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
> >>>>>>>>>>>
> >>>>>>>>>>>> driver has no knowledge about what the data is or where/if the data will
> >>>>>>>>>>>> be stored.
> >>>>>>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
> >>>>>>>>>> FYI: This is not strictly correct. In a coming product there is a
> >>>>>>>>>> case where the data will be stored in RAM. Richard Gong was also
> >>>>>>>>>> looking to use this driver to validate an image without programming
> >>>>>>>>>> or storing it. The fpga-image-load driver has no expectation that
> >>>>>>>>>> the data will be stored in nvmem, or even that it will be stored
> >>>>>>>>>> at all.
> >>>>>>>>> OK, we can discuss that use case then. But fundamentally a driver should
> >>>>>>>>> be clear what it is doing.
> >>>>>>>> The lower-level driver is, of course, clear what it is doing. And the
> >>>>>>>> FPGA Image Load Framework simply provides a consistent API and manages
> >>>>>>>> a potentially long-running data transfer in the context of a kernel
> >>>>>>>> worker thread.
> >>>>>>>>
> >>>>>>>> It sounds like you are saying that that is not "clear enough" in the
> >>>>>>>> context of the FPGA Manager?
> >>>>>>>>
> >>>>>>>> The files that are used with Intel PAC devices are self-describing. The
> >>>>>>>> user-space tools, the class driver and the lower-level driver just pass
> >>>>>>>> the data through to the card BMC without any knowledge of the content,
> >>>>>>>> purpose or final destination of the data.
> >>>>>>>>
> >>>>>>>> The card BMC will receive signed data, validate it, and process it as a
> >>>>>>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
> >>>>>>> I category all these actions as firmware update fully or partially on
> >>>>>>> persistent storage. The FPGA Manager don't have to know the meaning of
> >>>>>>> every byte on flash, but it should be aware the firmware is updated and
> >>>>>>> the card may acts differently with a new firmware. This is the common
> >>>>>>> working model for most of the FPGA cards so that we implement it in FPGA
> >>>>>>> manager class. 
> >>>>>>>
> >>>>>>>> the n6000, it could also be part of a multi-step process for programming
> >>>>>>>> SDM keys and the data may not be stored permanently.
> >>>>>>> This is new to me, but seems to be different from firmware update, so lets
> >>>>>>> think about it again.
> >>>>>>>
> >>>>>>>>> You may try to extend the FPGA framework to
> >>>>>>>>> support nvmem storage, or image validation, but cannot say we feed the
> >>>>>>>>> data to any engine undefined by the framework.
> >>>>>>>> I'm not sure what you mean by "feed the data to any engine undefined by the
> >>>>>>>> framework". I think the "engine" is the lower level driver/device that invokes
> >>>>>>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
> >>>>>>>> The fpga_mgr cannot control what driver invokes it.
> >>>>>>>>
> >>>>>>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
> >>>>>>>> data. Meaning that a self-describing file alone is not acceptable?
> >>>>>>> The class driver should define a reasonable working model and APIs.
> >>>>>>> Updating the FPGA backup storage is good to me. But receiving a mystery
> >>>>>>> box and do whatever it requires is not.
> >>>>>>>
> >>>>>>> Self-describing file is OK, encryption is OK, but either the class
> >>>>>>> driver itself, or with the help of the low level driver, should make
> >>>>>>> sure it works within its scope.
> >>>>>> In our secure update process, the card BMC firmware authenticates
> >>>>>> the data using the root entry hashes and will either reject the
> >>>>>> data or perform some function based on the contents. Neither the
> >>>>>> user-space, the class driver, nor the lower level driver know
> >>>>>> what the contents are. It _is_ a "mystery box" to them. How do we
> >>>>>> verify scope in this model?
> >>>>> I think we need to find out how. One case is, the HW is designed to have
> >>>>> one single function, such as firmware update, then any image input
> >>>>> through firmware update API is within expectation, and the driver
> >>>>> should only serve the firmware update API. I think this is how the
> >>>>> N3000 is working now. If the HW is for another function, register itself
> >>>>> to serve another API, or another class driver.
> >>>>>
> >>>>> Another case is, the HW could do multiple types of tasks depending on
> >>>>> the content of the image, such as firmware update, image verification,
> >>>>> or assumably power off the card ... There should be some mechanism for
> >>>>> the driver to only accept the right image according to what API is called.
> >>>>> Or the user may input an image named update_the_card.img through
> >>>>> firmware update API and finally get the card off. Having some headers
> >>>>> readable by host for the operation type? Or, some HW interface for host
> >>>>> to apply the operation type as well as the image, let the HW verify?
> >>>>> Let's think about it.
> >>>> I'm not sure if I am following your thinking here. The HW, of course,
> >>>> verifies authentication of the image and acts according to the image
> >>>> type. I don't think it is reasonable to require the class driver to
> >>>> interpret the data to determine what it is. That implies that the
> >>>> class driver would have to know the header format and possible values.
> >>>> It also means that changes to the header format would require patches
> >>>> to the class driver.
> >>>>
> >>>> The FPGA card is trusted by virtue of the fact that the customer
> >>>> purchased it and physically placed it in the machine. If the FPGA card
> >>>> (or the lower level driver) validates the image, then why does the
> >>>> Class driver need to be concerned about the file type? I think the
> >>>> purpose of the class driver is primarily to provide a common API and
> >>>> perform common functions so that they don't have to be replicated
> >>>> among similar low-level drivers. It is up to the low-level driver
> >>>> or the device itself to ensure that the data received is acceptable.
> >>>>
> >>>> If the card receives data through the fpga-mgr upload facility that
> >>>> isn't strictly a firmware update, and if the lower-level driver or
> >>>> the card handles it and returns appropriate status - is that really
> >>>> a problem?
> >>>>>> As you have noted, most current cases result in a change to the
> >>>>>> card, and I suspect that it will remain that way. But that can't be
> >>>>>> guaranteed, and I'm not convinced that a host driver needs to be
> >>>>>> concerned about it.
> >>>>> A host driver should know what is done, in some abstraction level.
> >>>>> I think updating the persistent storage is an acceptable abstraction
> >>>>> in FPGA domain, so I'd like to extend it in FPGA manager. But doing
> >>>>> anything according to the image is not.
> >>>> By host driver, do you mean the class driver? Or the lower-level device
> >>>> driver?
> >>> The class driver.
> >>>
> >>>> It seems to me that you are saying that self-describing images are not
> >>>> acceptable? Or at least they are not acceptable payload for an FPGA
> >>>> Manager firmware-update API?
> >>> For N3000, we are working on the persistent storage update APIs, which is
> >>> a much simpler working model, no runtime device change, and needs no
> >>> device removal & re-enumeration.
> >>>
> >>> But if you need to extend something more that would potentially changes
> >>> the behavior of the running devices on FPGA, device removal &
> >>> re-enumeration are needed so that the system knows what devices are
> >>> changed.
> >>>
> >>>> The FPGA Image Load Framework was designed with the concept of
> >>>> transferring data to a device without imposing a purpose on the data.
> >>>> The expectation is that the lower-level driver or the device will
> >>>> validate the data. Is there something fundamentally wrong with that
> >>> I think there is something wrong here. As I said before, persistent
> >>> storage updating has different software process from some runtime
> >>> updating, so the class driver should be aware of what the HW engine
> >>> is doing.
> >> So far, there are no self-describing images that cause a
> >> change in run-time behavior, and I don't think that will
> >> happen for the very reason that the class-driver would
> >> need to know about it.
> > Again, the class driver needs to know what is happening, at some
> > abstraction level, to ensure the system is aligned with the HW state.
> >
> > If the class driver cannot tell the detail, it has to assume the
> > whole FPGA region will be changed, and removal & re-enumeration is
> > needed.
> So we make it a requirement that the self-describing files
> cannot make changes that require the class driver to manage
> state.

The API should not only define what it won't do, but also define what
it will do. But the "image load" just specifies the top half of the
process. So I don't think this API would be accepted.

> >
> >> When I asserted that not all self-describing images are
> >> changing firmware, I did not mean to imply that they change
> >> run-time behavior; they do not. They are part of a multi-
> >> step update of firmware. However, looking at each part of
> >> the sequence independently, there is at least one instance
> >> where a certificate is stored in RAM for temporary use.
> >> When the driver exits from this call, there is no firmware
> >> change. There is also no change in run-time behavior.
> >>
> >> I'm thinking we could have different IOCTLs:
> >>
> >> (1) firmware  update (address, size, purpose provided
> >>     with the image)
> > Will the firmware update use the self-describing files?
> The firmware update option would be for files that
> are not self-describing.
> >
> >> (2) image upload (self-describing files)
> > If both 1 & 2 use self-describing files, how the class driver verifies
> > the type of the file without looking into the file?
> Only 2 would use self-describing files.

Which IOCTL the N3000 flash secure update will use?

Thanks,
Yilun

> 
> - Russ
> >
> > For example, if a user calls a firmware update API but inputs an image
> > upload file, will the class driver block the call? How?
> >
> >> (3) image validation
> >>
> >> These would all use most of the same code, but the FPGA
> >> Manager flags and structure fields would be set differently.
> > This is good to me.
> >
> > Thanks,
> > Yilun
> >
> >> - Russ
> >>> Thanks,
> >>> Yilun
> >>>
> >>>> approach? And if not, why couldn't we incorporate a similar image_load
> >>>> API into the FPGA Manager?
> >>>>
> >>>> - Russ
> >>>>
> >>>>> Thanks,
> >>>>> Yilun
> >>>>>
> >>>>>> - Russ
> >>>>>>
> >>>>>>> Thanks,
> >>>>>>> Yilun
> >>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> - Russ
> >>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Yilun
> >>>>>>>>>
> >>>>>>>>>>> while
> >>>>>>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
> >>>>>>>>>>> to know the exact physical position to store, may not, depends on what the
> >>>>>>>>>>> HW engines are.
> >>>>>>>>>>>
> >>>>>>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
> >>>>>>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
> >>>>>>>>>>>> See this email for a recap:
> >>>>>>>>>>>>
> >>>>>>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
> >>>>>>>>>>>>
> >>>>>>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
> >>>>>>>>>>>> approach probably fits better than the sysfs implementation. At the time
> >>>>>>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
> >>>>>>>>>>> I think at least developers don't have to go through 2 sets of software
> >>>>>>>>>>> stacks which are of the same concept. And adding some new features like
> >>>>>>>>>>> optionally threading or canceling data transfer are also good to FPGA
> >>>>>>>>>>> region update. And the nvmem update could also be benifit from exsiting
> >>>>>>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
> >>>>>>>>>>>
> >>>>>>>>>>> I try to explain myself according to each of your concern from that mail
> >>>>>>>>>>> thread:
> >>>>>>>>>>>
> >>>>>>>>>>> Purpose of the 2 updates
> >>>>>>>>>>> ========================
> >>>>>>>>>>>
> >>>>>>>>>>>   As I said before, I think they are both data transfer devices. FPGA
> >>>>>>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
> >>>>>>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
> >>>>>>>>>>>
> >>>>>>>>>>> Extra APIs that are unique to nvmem update
> >>>>>>>>>>> ==========================================
> >>>>>>>>>>>
> >>>>>>>>>>>   cdev APIs for nvmem update:
> >>>>>>>>>>>     Yes we need to expand the functionality so we need them.
> >>>>>>>>>>>
> >>>>>>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
> >>>>>>>>>>>   region:
> >>>>>>>>>>>     These are features in later patchsets which are not submitted, but we
> >>>>>>>>>>>     could talk about it in advance. I think this is actually a FPGA region
> >>>>>>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
> >>>>>>>>>>>     and re-enumeration, or a single command to image_load HW may result system
> >>>>>>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
> >>>>>>>>>>>     user data, maybe we add support for update from nvmem later.
> >>>>>>>>>>>
> >>>>>>>>>>>   fpga-mgr state extend:
> >>>>>>>>>>>     I think it could be extended, The current states are not perfect,
> >>>>>>>>>>>     adding something like IDLE or READY is just fine.
> >>>>>>>>>>>
> >>>>>>>>>>>   fpga-mgr status extend:
> >>>>>>>>>>>     Add general error definitions as needed. If there is some status
> >>>>>>>>>>>     that is quite vendor specific, expose it in low-level driver.
> >>>>>>>>>>>
> >>>>>>>>>>>   remaining-size:
> >>>>>>>>>>>     Nice to have for all.
> >>>>>>>>>>>
> >>>>>>>>>>> Threading the update
> >>>>>>>>>>> ====================
> >>>>>>>>>>>
> >>>>>>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
> >>>>>>>>>>>   reprogrammer?
> >>>>>>>>>>>
> >>>>>>>>>>> Cancelling the update
> >>>>>>>>>>> ====================
> >>>>>>>>>>>
> >>>>>>>>>>>   Also a good option for FPGA region update if HW engine supports.
> >>>>>>>>>> These are all good points.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> - Russ
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Yilun
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
> >>>>>>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
> >>>>>>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
> >>>>>>>>>>>>>>>> differences like threading or canceling the update, which are
> >>>>>>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
> >>>>>>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
> >>>>>>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
> >>>>>>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
> >>>>>>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
> >>>>>>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
> >>>>>>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
> >>>>>>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
> >>>>>>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
> >>>>>>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
> >>>>>>>>>>>>>>>> another topic.
> >>>>>>>>>>>> I'll give this some more thought and see if I can come up with some RFC
> >>>>>>>>>>>> patches.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Russ
> >>>>>>>>>>>>>>>> More discussion is appreciated.
> >>>>>>>>>>>>>>> I also think fpga_mgr could be extended.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In this patchset,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> new bit/flag was added to fpga_image_info
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
> >>>>>>>>>>>>>>> this patchset.
> >>>>>>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
> >>>>>>>>>>>>>> the 2 processes are almost the same.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
> >>>>>>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
> >>>>>>>>>>>>>> flags.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> How do you think?
> >>>>>>>>>>>>> A single set is fine.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Tom
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Yilun
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Any other driver would do similar.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Is this close to what you are thinking ?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Tom
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> Yilun
> >>>>>>>>>>>>>>>>

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/19/21 6:16 PM, Xu Yilun wrote:
> On Tue, Oct 19, 2021 at 08:09:46AM -0700, Russ Weight wrote:
>>
>> On 10/18/21 7:53 PM, Xu Yilun wrote:
>>> On Mon, Oct 18, 2021 at 09:24:08AM -0700, Russ Weight wrote:
>>>> On 10/18/21 1:13 AM, Xu Yilun wrote:
>>>>> On Fri, Oct 15, 2021 at 10:34:23AM -0700, Russ Weight wrote:
>>>>>> On 10/14/21 7:51 PM, Xu Yilun wrote:
>>>>>>> On Thu, Oct 14, 2021 at 09:32:53AM -0700, Russ Weight wrote:
>>>>>>>> On 10/13/21 6:49 PM, Xu Yilun wrote:
>>>>>>>>> On Wed, Oct 13, 2021 at 11:09:08AM -0700, Russ Weight wrote:
>>>>>>>>>> On 10/12/21 6:06 PM, Xu Yilun wrote:
>>>>>>>>>>> On Tue, Oct 12, 2021 at 10:20:15AM -0700, Russ Weight wrote:
>>>>>>>>>>>> On 10/12/21 12:47 AM, Xu Yilun wrote:
>>>>>>>>>>>>> On Mon, Oct 11, 2021 at 06:00:16PM -0700, Russ Weight wrote:
>>>>>>>>>>>>>> On 10/11/21 5:35 AM, Tom Rix wrote:
>>>>>>>>>>>>>>> On 10/10/21 6:41 PM, Xu Yilun wrote:
>>>>>>>>>>>>>>>> On Sat, Oct 09, 2021 at 05:11:20AM -0700, Tom Rix wrote:
>>>>>>>>>>>>>>>>> On 10/9/21 1:08 AM, Xu Yilun wrote:
>>>>>>>>>>>>>>>>>> On Wed, Sep 29, 2021 at 04:00:20PM -0700, Russ Weight wrote:
>>>>>>>>>>>>>>>>>>> The FPGA Image Load framework provides an API to upload image
>>>>>>>>>>>>>>>>>>> files to an FPGA device. Image files are self-describing. They could
>>>>>>>>>>>>>>>>>>> contain FPGA images, BMC images, Root Entry Hashes, or other device
>>>>>>>>>>>>>>>>>>> specific files. It is up to the lower-level device driver and the
>>>>>>>>>>>>>>>>>>> target device to authenticate and disposition the file data.
>>>>>>>>>>>>>>>>>> I've reconsider the FPGA persistent image update again, and think we
>>>>>>>>>>>>>>>>>> may include it in FPGA manager framework.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sorry I raised this topic again when it is already at patch v17, but now
>>>>>>>>>>>>>>>>>> I need to consider more seriously than before.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We have consensus the FPGA persistent image update is just like a normal
>>>>>>>>>>>>>>>>>> firmware update which finally writes the nvmem like flash or eeprom,
>>>>>>>>>>>>>>>>>> while the current FPGA manager deals with the active FPGA region update
>>>>>>>>>>>>>>>>>> and re-activation. Could we just expand the FPGA manager and let it handle
>>>>>>>>>>>>>>>>>> the nvmem update as well? Many FPGA cards have nvmem and downloaders
>>>>>>>>>>>>>>>>>> supports updating both FPGA region and nvmem.
>>>>>>>>>>>>>> The fpga-image-load driver is actually just a data transfer. The class
>>>>>>>>>>>>> IMHO, The fpga-mgr dev is also a data transfer. The fpga-region dev is
>>>>>>>>>>>>> acting as the FPGA region admin responsible for gating, transfering and
>>>>>>>>>>>>> re-enumerating.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So my opinion is to add a new data transfer type and keep a unified process.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> driver has no knowledge about what the data is or where/if the data will
>>>>>>>>>>>>>> be stored.
>>>>>>>>>>>>> The fpga-image-load driver knows the data will be stored in nvmem,
>>>>>>>>>>>> FYI: This is not strictly correct. In a coming product there is a
>>>>>>>>>>>> case where the data will be stored in RAM. Richard Gong was also
>>>>>>>>>>>> looking to use this driver to validate an image without programming
>>>>>>>>>>>> or storing it. The fpga-image-load driver has no expectation that
>>>>>>>>>>>> the data will be stored in nvmem, or even that it will be stored
>>>>>>>>>>>> at all.
>>>>>>>>>>> OK, we can discuss that use case then. But fundamentally a driver should
>>>>>>>>>>> be clear what it is doing.
>>>>>>>>>> The lower-level driver is, of course, clear what it is doing. And the
>>>>>>>>>> FPGA Image Load Framework simply provides a consistent API and manages
>>>>>>>>>> a potentially long-running data transfer in the context of a kernel
>>>>>>>>>> worker thread.
>>>>>>>>>>
>>>>>>>>>> It sounds like you are saying that that is not "clear enough" in the
>>>>>>>>>> context of the FPGA Manager?
>>>>>>>>>>
>>>>>>>>>> The files that are used with Intel PAC devices are self-describing. The
>>>>>>>>>> user-space tools, the class driver and the lower-level driver just pass
>>>>>>>>>> the data through to the card BMC without any knowledge of the content,
>>>>>>>>>> purpose or final destination of the data.
>>>>>>>>>>
>>>>>>>>>> The card BMC will receive signed data, validate it, and process it as a
>>>>>>>>>> BMC image, an FPGA image, a Root Entry Hash, or a key cancellation. In
>>>>>>>>> I category all these actions as firmware update fully or partially on
>>>>>>>>> persistent storage. The FPGA Manager don't have to know the meaning of
>>>>>>>>> every byte on flash, but it should be aware the firmware is updated and
>>>>>>>>> the card may acts differently with a new firmware. This is the common
>>>>>>>>> working model for most of the FPGA cards so that we implement it in FPGA
>>>>>>>>> manager class. 
>>>>>>>>>
>>>>>>>>>> the n6000, it could also be part of a multi-step process for programming
>>>>>>>>>> SDM keys and the data may not be stored permanently.
>>>>>>>>> This is new to me, but seems to be different from firmware update, so lets
>>>>>>>>> think about it again.
>>>>>>>>>
>>>>>>>>>>> You may try to extend the FPGA framework to
>>>>>>>>>>> support nvmem storage, or image validation, but cannot say we feed the
>>>>>>>>>>> data to any engine undefined by the framework.
>>>>>>>>>> I'm not sure what you mean by "feed the data to any engine undefined by the
>>>>>>>>>> framework". I think the "engine" is the lower level driver/device that invokes
>>>>>>>>>> the fpga_mgr. The lower level driver, of course, is clear what it is doing.
>>>>>>>>>> The fpga_mgr cannot control what driver invokes it.
>>>>>>>>>>
>>>>>>>>>> Are saying that when invoking the fpga-mgr, that it _must_ also pass descriptive
>>>>>>>>>> data. Meaning that a self-describing file alone is not acceptable?
>>>>>>>>> The class driver should define a reasonable working model and APIs.
>>>>>>>>> Updating the FPGA backup storage is good to me. But receiving a mystery
>>>>>>>>> box and do whatever it requires is not.
>>>>>>>>>
>>>>>>>>> Self-describing file is OK, encryption is OK, but either the class
>>>>>>>>> driver itself, or with the help of the low level driver, should make
>>>>>>>>> sure it works within its scope.
>>>>>>>> In our secure update process, the card BMC firmware authenticates
>>>>>>>> the data using the root entry hashes and will either reject the
>>>>>>>> data or perform some function based on the contents. Neither the
>>>>>>>> user-space, the class driver, nor the lower level driver know
>>>>>>>> what the contents are. It _is_ a "mystery box" to them. How do we
>>>>>>>> verify scope in this model?
>>>>>>> I think we need to find out how. One case is, the HW is designed to have
>>>>>>> one single function, such as firmware update, then any image input
>>>>>>> through firmware update API is within expectation, and the driver
>>>>>>> should only serve the firmware update API. I think this is how the
>>>>>>> N3000 is working now. If the HW is for another function, register itself
>>>>>>> to serve another API, or another class driver.
>>>>>>>
>>>>>>> Another case is, the HW could do multiple types of tasks depending on
>>>>>>> the content of the image, such as firmware update, image verification,
>>>>>>> or assumably power off the card ... There should be some mechanism for
>>>>>>> the driver to only accept the right image according to what API is called.
>>>>>>> Or the user may input an image named update_the_card.img through
>>>>>>> firmware update API and finally get the card off. Having some headers
>>>>>>> readable by host for the operation type? Or, some HW interface for host
>>>>>>> to apply the operation type as well as the image, let the HW verify?
>>>>>>> Let's think about it.
>>>>>> I'm not sure if I am following your thinking here. The HW, of course,
>>>>>> verifies authentication of the image and acts according to the image
>>>>>> type. I don't think it is reasonable to require the class driver to
>>>>>> interpret the data to determine what it is. That implies that the
>>>>>> class driver would have to know the header format and possible values.
>>>>>> It also means that changes to the header format would require patches
>>>>>> to the class driver.
>>>>>>
>>>>>> The FPGA card is trusted by virtue of the fact that the customer
>>>>>> purchased it and physically placed it in the machine. If the FPGA card
>>>>>> (or the lower level driver) validates the image, then why does the
>>>>>> Class driver need to be concerned about the file type? I think the
>>>>>> purpose of the class driver is primarily to provide a common API and
>>>>>> perform common functions so that they don't have to be replicated
>>>>>> among similar low-level drivers. It is up to the low-level driver
>>>>>> or the device itself to ensure that the data received is acceptable.
>>>>>>
>>>>>> If the card receives data through the fpga-mgr upload facility that
>>>>>> isn't strictly a firmware update, and if the lower-level driver or
>>>>>> the card handles it and returns appropriate status - is that really
>>>>>> a problem?
>>>>>>>> As you have noted, most current cases result in a change to the
>>>>>>>> card, and I suspect that it will remain that way. But that can't be
>>>>>>>> guaranteed, and I'm not convinced that a host driver needs to be
>>>>>>>> concerned about it.
>>>>>>> A host driver should know what is done, in some abstraction level.
>>>>>>> I think updating the persistent storage is an acceptable abstraction
>>>>>>> in FPGA domain, so I'd like to extend it in FPGA manager. But doing
>>>>>>> anything according to the image is not.
>>>>>> By host driver, do you mean the class driver? Or the lower-level device
>>>>>> driver?
>>>>> The class driver.
>>>>>
>>>>>> It seems to me that you are saying that self-describing images are not
>>>>>> acceptable? Or at least they are not acceptable payload for an FPGA
>>>>>> Manager firmware-update API?
>>>>> For N3000, we are working on the persistent storage update APIs, which is
>>>>> a much simpler working model, no runtime device change, and needs no
>>>>> device removal & re-enumeration.
>>>>>
>>>>> But if you need to extend something more that would potentially changes
>>>>> the behavior of the running devices on FPGA, device removal &
>>>>> re-enumeration are needed so that the system knows what devices are
>>>>> changed.
>>>>>
>>>>>> The FPGA Image Load Framework was designed with the concept of
>>>>>> transferring data to a device without imposing a purpose on the data.
>>>>>> The expectation is that the lower-level driver or the device will
>>>>>> validate the data. Is there something fundamentally wrong with that
>>>>> I think there is something wrong here. As I said before, persistent
>>>>> storage updating has different software process from some runtime
>>>>> updating, so the class driver should be aware of what the HW engine
>>>>> is doing.
>>>> So far, there are no self-describing images that cause a
>>>> change in run-time behavior, and I don't think that will
>>>> happen for the very reason that the class-driver would
>>>> need to know about it.
>>> Again, the class driver needs to know what is happening, at some
>>> abstraction level, to ensure the system is aligned with the HW state.
>>>
>>> If the class driver cannot tell the detail, it has to assume the
>>> whole FPGA region will be changed, and removal & re-enumeration is
>>> needed.
>> So we make it a requirement that the self-describing files
>> cannot make changes that require the class driver to manage
>> state.
> The API should not only define what it won't do, but also define what
> it will do. But the "image load" just specifies the top half of the
> process. So I don't think this API would be accepted.
So what is the path forward. It seems like you are saying
that the self-describing files do not fit in the fpga-mgr.
Can we reconsider the FPGA Image Load Framework, which does
not make any assumptions about the contents of the image
files?

>
>>>> When I asserted that not all self-describing images are
>>>> changing firmware, I did not mean to imply that they change
>>>> run-time behavior; they do not. They are part of a multi-
>>>> step update of firmware. However, looking at each part of
>>>> the sequence independently, there is at least one instance
>>>> where a certificate is stored in RAM for temporary use.
>>>> When the driver exits from this call, there is no firmware
>>>> change. There is also no change in run-time behavior.
>>>>
>>>> I'm thinking we could have different IOCTLs:
>>>>
>>>> (1) firmware  update (address, size, purpose provided
>>>>     with the image)
>>> Will the firmware update use the self-describing files?
>> The firmware update option would be for files that
>> are not self-describing.
>>>> (2) image upload (self-describing files)
>>> If both 1 & 2 use self-describing files, how the class driver verifies
>>> the type of the file without looking into the file?
>> Only 2 would use self-describing files.
> Which IOCTL the N3000 flash secure update will use?
It would use 2. All of our image update files are self-describing.

- Russ
>
> Thanks,
> Yilun
>
>> - Russ
>>> For example, if a user calls a firmware update API but inputs an image
>>> upload file, will the class driver block the call? How?
>>>
>>>> (3) image validation
>>>>
>>>> These would all use most of the same code, but the FPGA
>>>> Manager flags and structure fields would be set differently.
>>> This is good to me.
>>>
>>> Thanks,
>>> Yilun
>>>
>>>> - Russ
>>>>> Thanks,
>>>>> Yilun
>>>>>
>>>>>> approach? And if not, why couldn't we incorporate a similar image_load
>>>>>> API into the FPGA Manager?
>>>>>>
>>>>>> - Russ
>>>>>>
>>>>>>> Thanks,
>>>>>>> Yilun
>>>>>>>
>>>>>>>> - Russ
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Yilun
>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> - Russ
>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Yilun
>>>>>>>>>>>
>>>>>>>>>>>>> while
>>>>>>>>>>>>> the fpga-mgr knows the data will be stored in FPGA cells. They may need
>>>>>>>>>>>>> to know the exact physical position to store, may not, depends on what the
>>>>>>>>>>>>> HW engines are.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> This functionality could, of course, be merged into the fpga-mgr. I did
>>>>>>>>>>>>>> a proof of concept of this a while back and we discussed the pros and cons.
>>>>>>>>>>>>>> See this email for a recap:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://marc.info/?l=linux-fpga&m=161998085507374&w=2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Things have changed some with the evolution of the driver. The IOCTL
>>>>>>>>>>>>>> approach probably fits better than the sysfs implementation. At the time
>>>>>>>>>>>>>> it seemed that a merge would add unnecessary complexity without adding value.
>>>>>>>>>>>>> I think at least developers don't have to go through 2 sets of software
>>>>>>>>>>>>> stacks which are of the same concept. And adding some new features like
>>>>>>>>>>>>> optionally threading or canceling data transfer are also good to FPGA
>>>>>>>>>>>>> region update. And the nvmem update could also be benifit from exsiting
>>>>>>>>>>>>> implementations like scatter-gather buffers, in-kernel firmware loading.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I try to explain myself according to each of your concern from that mail
>>>>>>>>>>>>> thread:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Purpose of the 2 updates
>>>>>>>>>>>>> ========================
>>>>>>>>>>>>>
>>>>>>>>>>>>>   As I said before, I think they are both data transfer devices. FPGA
>>>>>>>>>>>>> region update gets extra support from fpga-region & fpga-bridge, and
>>>>>>>>>>>>> FPGA nvmem update could be a standalone fpga-mgr.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Extra APIs that are unique to nvmem update
>>>>>>>>>>>>> ==========================================
>>>>>>>>>>>>>
>>>>>>>>>>>>>   cdev APIs for nvmem update:
>>>>>>>>>>>>>     Yes we need to expand the functionality so we need them.
>>>>>>>>>>>>>
>>>>>>>>>>>>>   available_images, image_load APIs for loading nvmem content to FPGA
>>>>>>>>>>>>>   region:
>>>>>>>>>>>>>     These are features in later patchsets which are not submitted, but we
>>>>>>>>>>>>>     could talk about it in advance. I think this is actually a FPGA region
>>>>>>>>>>>>>     update from nvmem, it also requires gating, data loading (no SW transfer)
>>>>>>>>>>>>>     and re-enumeration, or a single command to image_load HW may result system
>>>>>>>>>>>>>     disordered. The FPGA framework now only supports update from in-kernel
>>>>>>>>>>>>>     user data, maybe we add support for update from nvmem later.
>>>>>>>>>>>>>
>>>>>>>>>>>>>   fpga-mgr state extend:
>>>>>>>>>>>>>     I think it could be extended, The current states are not perfect,
>>>>>>>>>>>>>     adding something like IDLE or READY is just fine.
>>>>>>>>>>>>>
>>>>>>>>>>>>>   fpga-mgr status extend:
>>>>>>>>>>>>>     Add general error definitions as needed. If there is some status
>>>>>>>>>>>>>     that is quite vendor specific, expose it in low-level driver.
>>>>>>>>>>>>>
>>>>>>>>>>>>>   remaining-size:
>>>>>>>>>>>>>     Nice to have for all.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Threading the update
>>>>>>>>>>>>> ====================
>>>>>>>>>>>>>
>>>>>>>>>>>>>   Also a good option for FPGA region update, maybe we also have a slow FPGA
>>>>>>>>>>>>>   reprogrammer?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cancelling the update
>>>>>>>>>>>>> ====================
>>>>>>>>>>>>>
>>>>>>>>>>>>>   Also a good option for FPGA region update if HW engine supports.
>>>>>>>>>>>> These are all good points.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> - Russ
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Yilun
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> According to the patchset, the basic workflow of the 2 update types are
>>>>>>>>>>>>>>>>>> quite similar, get the data, prepare for the HW, write and complete.
>>>>>>>>>>>>>>>>>> They are already implemented in FPGA manager. We've discussed some
>>>>>>>>>>>>>>>>>> differences like threading or canceling the update, which are
>>>>>>>>>>>>>>>>>> not provided by FPGA manager but they may also nice to have for FPGA
>>>>>>>>>>>>>>>>>> region update. An FPGA region update may also last for a long time??
>>>>>>>>>>>>>>>>>> So I think having 2 sets of similar frameworks in FPGA is unnecessary.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> My quick mind is that we add some flags in struct fpga_mgr & struct
>>>>>>>>>>>>>>>>>> fpga_image_info to indicate the HW capability (support FPGA region
>>>>>>>>>>>>>>>>>> update or nvmem update or both) of the download engine and the provided
>>>>>>>>>>>>>>>>>> image type. Then the low-level driver knows how to download if it
>>>>>>>>>>>>>>>>>> supports both image types.An char device could be added for each fpga manager dev, providing the
>>>>>>>>>>>>>>>>>> user APIs for nvmem update. We may not use the char dev for FPGA region
>>>>>>>>>>>>>>>>>> update cause it changes the system HW devices and needs device hotplug
>>>>>>>>>>>>>>>>>> in FPGA region. We'd better leave it to FPGA region class, this is
>>>>>>>>>>>>>>>>>> another topic.
>>>>>>>>>>>>>> I'll give this some more thought and see if I can come up with some RFC
>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Russ
>>>>>>>>>>>>>>>>>> More discussion is appreciated.
>>>>>>>>>>>>>>>>> I also think fpga_mgr could be extended.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In this patchset,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://lore.kernel.org/linux-fpga/20210625195849.837976-1-trix@redhat.com/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> A second, similar set of write ops was added to fpga_manger_ops,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> new bit/flag was added to fpga_image_info
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The intent was for dfl to add their specific ops to cover what is done in
>>>>>>>>>>>>>>>>> this patchset.
>>>>>>>>>>>>>>>> I think we don't have to add 2 ops for reconfig & reimage in framework,
>>>>>>>>>>>>>>>> the 2 processes are almost the same.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Just add the _REIMAGE (or something else, NVMEM?) flag for
>>>>>>>>>>>>>>>> fpga_image_info, and low level drivers handle it as they do for other
>>>>>>>>>>>>>>>> flags.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How do you think?
>>>>>>>>>>>>>>> A single set is fine.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> A difficult part of is the length of  time to do the write. The existing write should be improved to use a worker thread.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Yilun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any other driver would do similar.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Is this close to what you are thinking ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Yilun
>>>>>>>>>>>>>>>>>>

Re: [PATCH v17 4/5] fpga: image-load: add status ioctl

[Russ Weight]

On 10/15/21 1:22 PM, Lizhi Hou wrote:
>
> On 9/29/21 4:00 PM, Russ Weight wrote:
>> Extend the FPGA Image Load framework to include an FPGA_IMAGE_LOAD_STATUS
>> IOCTL that can be used to monitor the progress of an ongoing image upload.
>> The status returned includes how much data remains to be transferred, the
>> progress of the image upload, and error information in the case of a
>> failure.
>>
>> Signed-off-by: Russ Weight <russell.h.weight@intel.com>
>> ---
>> v17:
>>   - Rebased for changes to earlier patches.
>> v16:
>>   - Minor changes to adapt in changes in prevoius patches.
>> v15:
>>   - This patch is new to the patchset and provides an FPGA_IMAGE_LOAD_STATUS
>>     IOCTL to return the current values for: remaining_size, progress,
>>     err_progress, and err_code.
>>   - This patch has elements of the following three patches from the previous
>>     patch-set:
>>       [PATCH v14 3/6] fpga: sec-mgr: expose sec-mgr update status
>>       [PATCH v14 4/6] fpga: sec-mgr: expose sec-mgr update errors
>>       [PATCH v14 5/6] fpga: sec-mgr: expose sec-mgr update size
>>   - Changed file, symbol, and config names to reflect the new driver name
>>   - There are some minor changes to locking to enable this ioctl to return
>>     coherent data.
>> ---
>>   Documentation/fpga/fpga-image-load.rst |  6 +++
>>   drivers/fpga/fpga-image-load.c         | 58 +++++++++++++++++++++-----
>>   include/linux/fpga/fpga-image-load.h   |  1 +
>>   include/uapi/linux/fpga-image-load.h   | 18 ++++++++
>>   4 files changed, 73 insertions(+), 10 deletions(-)
>>
>> diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
>> index 487b5466f67c..f64f5ee473b8 100644
>> --- a/Documentation/fpga/fpga-image-load.rst
>> +++ b/Documentation/fpga/fpga-image-load.rst
>> @@ -33,3 +33,9 @@ being updated. This is an exclusive operation; an attempt to start
>>   concurrent image uploads for the same device will fail with EBUSY. An
>>   eventfd file descriptor parameter is provided to this IOCTL. It will be
>>   signalled at the completion of the image upload.
>> +
>> +FPGA_IMAGE_LOAD_STATUS:
>> +
>> +Collect status for an on-going image upload. The status returned includes
>> +how much data remains to be transferred, the progress of the image upload,
>> +and error information in the case of a failure.
>> diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
>> index f04dfc71c190..58373b9e8c02 100644
>> --- a/drivers/fpga/fpga-image-load.c
>> +++ b/drivers/fpga/fpga-image-load.c
>> @@ -22,6 +22,22 @@ static dev_t fpga_image_devt;
>>
>>   #define to_image_load(d) container_of(d, struct fpga_image_load, dev)
>>
>> +static void fpga_image_update_progress(struct fpga_image_load *imgld,
>> +                                      u32 new_progress)
>> +{
>> +       mutex_lock(&imgld->lock);
>> +       imgld->progress = new_progress;
>> +       mutex_unlock(&imgld->lock);
>> +}
>> +
>> +static void fpga_image_set_error(struct fpga_image_load *imgld, u32 err_code)
>> +{
>> +       mutex_lock(&imgld->lock);
>> +       imgld->err_progress = imgld->progress;
>> +       imgld->err_code = err_code;
>> +       mutex_unlock(&imgld->lock);
>> +}
>> +
>>   static void fpga_image_prog_complete(struct fpga_image_load *imgld)
>>   {
>>          mutex_lock(&imgld->lock);
>> @@ -38,24 +54,24 @@ static void fpga_image_do_load(struct work_struct *work)
>>          imgld = container_of(work, struct fpga_image_load, work);
>>
>>          if (imgld->driver_unload) {
>> -               imgld->err_code = FPGA_IMAGE_ERR_CANCELED;
>> +               fpga_image_set_error(imgld, FPGA_IMAGE_ERR_CANCELED);
>>                  goto idle_exit;
>>          }
>>
>>          get_device(&imgld->dev);
>>          if (!try_module_get(imgld->dev.parent->driver->owner)) {
>> -               imgld->err_code = FPGA_IMAGE_ERR_BUSY;
>> +               fpga_image_set_error(imgld, FPGA_IMAGE_ERR_BUSY);
>>                  goto putdev_exit;
>>          }
>>
>> -       imgld->progress = FPGA_IMAGE_PROG_PREPARING;
>> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PREPARING);
>>          ret = imgld->ops->prepare(imgld, imgld->data, imgld->remaining_size);
>>          if (ret) {
>> -               imgld->err_code = ret;
>> +               fpga_image_set_error(imgld, ret);
>>                  goto modput_exit;
>>          }
>>
>> -       imgld->progress = FPGA_IMAGE_PROG_WRITING;
>> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_WRITING);
>>          while (imgld->remaining_size) {
>>                  ret = imgld->ops->write(imgld, imgld->data, offset,
>>                                          imgld->remaining_size);
>> @@ -65,7 +81,7 @@ static void fpga_image_do_load(struct work_struct *work)
>>                                           "write-op wrote zero data\n");
>>                                  ret = -FPGA_IMAGE_ERR_RW_ERROR;
>>                          }
>> -                       imgld->err_code = -ret;
>> +                       fpga_image_set_error(imgld, -ret);
>>                          goto done;
>>                  }
>>
>> @@ -73,10 +89,10 @@ static void fpga_image_do_load(struct work_struct *work)
>>                  offset += ret;
>>          }
>>
>> -       imgld->progress = FPGA_IMAGE_PROG_PROGRAMMING;
>> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PROGRAMMING);
>>          ret = imgld->ops->poll_complete(imgld);
>>          if (ret)
>> -               imgld->err_code = ret;
>> +               fpga_image_set_error(imgld, ret);
>>
>>   done:
>>          if (imgld->ops->cleanup)
>> @@ -151,20 +167,42 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
>>          return ret;
>>   }
>>
>> +static int fpga_image_load_ioctl_status(struct fpga_image_load *imgld,
>> +                                       unsigned long arg)
>> +{
>> +       struct fpga_image_status status;
>> +
>> +       memset(&status, 0, sizeof(status));
>> +       status.progress = imgld->progress;
>> +       status.remaining_size = imgld->remaining_size;
>> +       status.err_progress = imgld->err_progress;
>> +       status.err_code = imgld->err_code;
>> +
>> +       if (copy_to_user((void __user *)arg, &status, sizeof(status)))
>> +               return -EFAULT;
>> +
>> +       return 0;
>> +}
>> +
>>   static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
>>                                    unsigned long arg)
>>   {
>>          struct fpga_image_load *imgld = filp->private_data;
>>          int ret = -ENOTTY;
>>
>> +       mutex_lock(&imgld->lock);
>> +
>>          switch (cmd) {
>>          case FPGA_IMAGE_LOAD_WRITE:
>> -               mutex_lock(&imgld->lock);
>>                  ret = fpga_image_load_ioctl_write(imgld, arg);
>> -               mutex_unlock(&imgld->lock);
>> +               break;
>> +       case FPGA_IMAGE_LOAD_STATUS:
>> +               ret = fpga_image_load_ioctl_status(imgld, arg);
>>                  break;
>>          }
>>
>> +       mutex_unlock(&imgld->lock);
>> +
>>          return ret;
>>   }
>>
>> diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
>> index 77b3c91ce073..366111d090fb 100644
>> --- a/include/linux/fpga/fpga-image-load.h
>> +++ b/include/linux/fpga/fpga-image-load.h
>> @@ -49,6 +49,7 @@ struct fpga_image_load {
>>          const u8 *data;                 /* pointer to update data */
>>          u32 remaining_size;             /* size remaining to transfer */
>>          u32 progress;
>> +       u32 err_progress;               /* progress at time of error */
>>          u32 err_code;                   /* image load error code */
>>          bool driver_unload;
>>          struct eventfd_ctx *finished;
>> diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
>> index 8d2d3db92e87..1b91343961df 100644
>> --- a/include/uapi/linux/fpga-image-load.h
>> +++ b/include/uapi/linux/fpga-image-load.h
>> @@ -51,4 +51,22 @@ struct fpga_image_write {
>>
>>   #define FPGA_IMAGE_LOAD_WRITE  _IOW(FPGA_IMAGE_LOAD_MAGIC, 0, struct fpga_image_write)
>>
>> +/**
>> + * FPGA_IMAGE_LOAD_STATUS - _IOR(FPGA_IMAGE_LOAD_MAGIC, 1,
>> + *                              struct fpga_image_status)
>> + *
>> + * Request status information for an ongoing update.
>> + *
>> + * Return: 0 on success, -errno on failure.
>> + */
>> +struct fpga_image_status {
>> +       /* Output */
>> +       __u32 remaining_size;   /* size remaining to transfer */
>> +       __u32 progress;         /* current progress of image load */
>> +       __u32 err_progress;     /* progress at time of error */
>> +       __u32 err_code;         /* error code */
>> +};
>
> Could this be extended to also collect the image detail?
>
>         Image version, name, etc been successfully written to device (flash).
>
>         Image version, name, etc is currently running on the device.
>
> Or maybe add another query command to do these?
>
> So the userland utility will be able to show what image is running and what image is going to run with next cold reboot.

This status call is intended specifically to show the progress and
status of the image load. Yilun's comments on patch 5 suggest that
some of these types of operations could be handled as part of the
fpga-region class driver

I have patches that have not been submitted yet that contain some
of these types of features, although not in the class driver (yet).
Here is a link to documentation for what I have now. Take a look
at the sysfs node descriptions that are in the control subdirectory.
These are listed at the bottom of the file.

https://github.com/OPAE/linux-dfl/blob/fpga-ofs-dev/Documentation/ABI/testing/sysfs-driver-intel-m10-bmc-sec-update

- Russ


>
>
> Thanks,
>
> Lizhi
>
>> +
>> +#define FPGA_IMAGE_LOAD_STATUS _IOR(FPGA_IMAGE_LOAD_MAGIC, 1, struct fpga_image_status)
>> +
>>   #endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */
>> -- 
>> 2.25.1
>>

Re: [PATCH v17 4/5] fpga: image-load: add status ioctl

[Lizhi Hou]

On 10/20/21 2:42 PM, Russ Weight wrote:
> On 10/15/21 1:22 PM, Lizhi Hou wrote:
>> On 9/29/21 4:00 PM, Russ Weight wrote:
>>> Extend the FPGA Image Load framework to include an FPGA_IMAGE_LOAD_STATUS
>>> IOCTL that can be used to monitor the progress of an ongoing image upload.
>>> The status returned includes how much data remains to be transferred, the
>>> progress of the image upload, and error information in the case of a
>>> failure.
>>>
>>> Signed-off-by: Russ Weight <russell.h.weight@intel.com>
>>> ---
>>> v17:
>>>    - Rebased for changes to earlier patches.
>>> v16:
>>>    - Minor changes to adapt in changes in prevoius patches.
>>> v15:
>>>    - This patch is new to the patchset and provides an FPGA_IMAGE_LOAD_STATUS
>>>      IOCTL to return the current values for: remaining_size, progress,
>>>      err_progress, and err_code.
>>>    - This patch has elements of the following three patches from the previous
>>>      patch-set:
>>>        [PATCH v14 3/6] fpga: sec-mgr: expose sec-mgr update status
>>>        [PATCH v14 4/6] fpga: sec-mgr: expose sec-mgr update errors
>>>        [PATCH v14 5/6] fpga: sec-mgr: expose sec-mgr update size
>>>    - Changed file, symbol, and config names to reflect the new driver name
>>>    - There are some minor changes to locking to enable this ioctl to return
>>>      coherent data.
>>> ---
>>>    Documentation/fpga/fpga-image-load.rst |  6 +++
>>>    drivers/fpga/fpga-image-load.c         | 58 +++++++++++++++++++++-----
>>>    include/linux/fpga/fpga-image-load.h   |  1 +
>>>    include/uapi/linux/fpga-image-load.h   | 18 ++++++++
>>>    4 files changed, 73 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/Documentation/fpga/fpga-image-load.rst b/Documentation/fpga/fpga-image-load.rst
>>> index 487b5466f67c..f64f5ee473b8 100644
>>> --- a/Documentation/fpga/fpga-image-load.rst
>>> +++ b/Documentation/fpga/fpga-image-load.rst
>>> @@ -33,3 +33,9 @@ being updated. This is an exclusive operation; an attempt to start
>>>    concurrent image uploads for the same device will fail with EBUSY. An
>>>    eventfd file descriptor parameter is provided to this IOCTL. It will be
>>>    signalled at the completion of the image upload.
>>> +
>>> +FPGA_IMAGE_LOAD_STATUS:
>>> +
>>> +Collect status for an on-going image upload. The status returned includes
>>> +how much data remains to be transferred, the progress of the image upload,
>>> +and error information in the case of a failure.
>>> diff --git a/drivers/fpga/fpga-image-load.c b/drivers/fpga/fpga-image-load.c
>>> index f04dfc71c190..58373b9e8c02 100644
>>> --- a/drivers/fpga/fpga-image-load.c
>>> +++ b/drivers/fpga/fpga-image-load.c
>>> @@ -22,6 +22,22 @@ static dev_t fpga_image_devt;
>>>
>>>    #define to_image_load(d) container_of(d, struct fpga_image_load, dev)
>>>
>>> +static void fpga_image_update_progress(struct fpga_image_load *imgld,
>>> +                                      u32 new_progress)
>>> +{
>>> +       mutex_lock(&imgld->lock);
>>> +       imgld->progress = new_progress;
>>> +       mutex_unlock(&imgld->lock);
>>> +}
>>> +
>>> +static void fpga_image_set_error(struct fpga_image_load *imgld, u32 err_code)
>>> +{
>>> +       mutex_lock(&imgld->lock);
>>> +       imgld->err_progress = imgld->progress;
>>> +       imgld->err_code = err_code;
>>> +       mutex_unlock(&imgld->lock);
>>> +}
>>> +
>>>    static void fpga_image_prog_complete(struct fpga_image_load *imgld)
>>>    {
>>>           mutex_lock(&imgld->lock);
>>> @@ -38,24 +54,24 @@ static void fpga_image_do_load(struct work_struct *work)
>>>           imgld = container_of(work, struct fpga_image_load, work);
>>>
>>>           if (imgld->driver_unload) {
>>> -               imgld->err_code = FPGA_IMAGE_ERR_CANCELED;
>>> +               fpga_image_set_error(imgld, FPGA_IMAGE_ERR_CANCELED);
>>>                   goto idle_exit;
>>>           }
>>>
>>>           get_device(&imgld->dev);
>>>           if (!try_module_get(imgld->dev.parent->driver->owner)) {
>>> -               imgld->err_code = FPGA_IMAGE_ERR_BUSY;
>>> +               fpga_image_set_error(imgld, FPGA_IMAGE_ERR_BUSY);
>>>                   goto putdev_exit;
>>>           }
>>>
>>> -       imgld->progress = FPGA_IMAGE_PROG_PREPARING;
>>> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PREPARING);
>>>           ret = imgld->ops->prepare(imgld, imgld->data, imgld->remaining_size);
>>>           if (ret) {
>>> -               imgld->err_code = ret;
>>> +               fpga_image_set_error(imgld, ret);
>>>                   goto modput_exit;
>>>           }
>>>
>>> -       imgld->progress = FPGA_IMAGE_PROG_WRITING;
>>> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_WRITING);
>>>           while (imgld->remaining_size) {
>>>                   ret = imgld->ops->write(imgld, imgld->data, offset,
>>>                                           imgld->remaining_size);
>>> @@ -65,7 +81,7 @@ static void fpga_image_do_load(struct work_struct *work)
>>>                                            "write-op wrote zero data\n");
>>>                                   ret = -FPGA_IMAGE_ERR_RW_ERROR;
>>>                           }
>>> -                       imgld->err_code = -ret;
>>> +                       fpga_image_set_error(imgld, -ret);
>>>                           goto done;
>>>                   }
>>>
>>> @@ -73,10 +89,10 @@ static void fpga_image_do_load(struct work_struct *work)
>>>                   offset += ret;
>>>           }
>>>
>>> -       imgld->progress = FPGA_IMAGE_PROG_PROGRAMMING;
>>> +       fpga_image_update_progress(imgld, FPGA_IMAGE_PROG_PROGRAMMING);
>>>           ret = imgld->ops->poll_complete(imgld);
>>>           if (ret)
>>> -               imgld->err_code = ret;
>>> +               fpga_image_set_error(imgld, ret);
>>>
>>>    done:
>>>           if (imgld->ops->cleanup)
>>> @@ -151,20 +167,42 @@ static int fpga_image_load_ioctl_write(struct fpga_image_load *imgld,
>>>           return ret;
>>>    }
>>>
>>> +static int fpga_image_load_ioctl_status(struct fpga_image_load *imgld,
>>> +                                       unsigned long arg)
>>> +{
>>> +       struct fpga_image_status status;
>>> +
>>> +       memset(&status, 0, sizeof(status));
>>> +       status.progress = imgld->progress;
>>> +       status.remaining_size = imgld->remaining_size;
>>> +       status.err_progress = imgld->err_progress;
>>> +       status.err_code = imgld->err_code;
>>> +
>>> +       if (copy_to_user((void __user *)arg, &status, sizeof(status)))
>>> +               return -EFAULT;
>>> +
>>> +       return 0;
>>> +}
>>> +
>>>    static long fpga_image_load_ioctl(struct file *filp, unsigned int cmd,
>>>                                     unsigned long arg)
>>>    {
>>>           struct fpga_image_load *imgld = filp->private_data;
>>>           int ret = -ENOTTY;
>>>
>>> +       mutex_lock(&imgld->lock);
>>> +
>>>           switch (cmd) {
>>>           case FPGA_IMAGE_LOAD_WRITE:
>>> -               mutex_lock(&imgld->lock);
>>>                   ret = fpga_image_load_ioctl_write(imgld, arg);
>>> -               mutex_unlock(&imgld->lock);
>>> +               break;
>>> +       case FPGA_IMAGE_LOAD_STATUS:
>>> +               ret = fpga_image_load_ioctl_status(imgld, arg);
>>>                   break;
>>>           }
>>>
>>> +       mutex_unlock(&imgld->lock);
>>> +
>>>           return ret;
>>>    }
>>>
>>> diff --git a/include/linux/fpga/fpga-image-load.h b/include/linux/fpga/fpga-image-load.h
>>> index 77b3c91ce073..366111d090fb 100644
>>> --- a/include/linux/fpga/fpga-image-load.h
>>> +++ b/include/linux/fpga/fpga-image-load.h
>>> @@ -49,6 +49,7 @@ struct fpga_image_load {
>>>           const u8 *data;                 /* pointer to update data */
>>>           u32 remaining_size;             /* size remaining to transfer */
>>>           u32 progress;
>>> +       u32 err_progress;               /* progress at time of error */
>>>           u32 err_code;                   /* image load error code */
>>>           bool driver_unload;
>>>           struct eventfd_ctx *finished;
>>> diff --git a/include/uapi/linux/fpga-image-load.h b/include/uapi/linux/fpga-image-load.h
>>> index 8d2d3db92e87..1b91343961df 100644
>>> --- a/include/uapi/linux/fpga-image-load.h
>>> +++ b/include/uapi/linux/fpga-image-load.h
>>> @@ -51,4 +51,22 @@ struct fpga_image_write {
>>>
>>>    #define FPGA_IMAGE_LOAD_WRITE  _IOW(FPGA_IMAGE_LOAD_MAGIC, 0, struct fpga_image_write)
>>>
>>> +/**
>>> + * FPGA_IMAGE_LOAD_STATUS - _IOR(FPGA_IMAGE_LOAD_MAGIC, 1,
>>> + *                              struct fpga_image_status)
>>> + *
>>> + * Request status information for an ongoing update.
>>> + *
>>> + * Return: 0 on success, -errno on failure.
>>> + */
>>> +struct fpga_image_status {
>>> +       /* Output */
>>> +       __u32 remaining_size;   /* size remaining to transfer */
>>> +       __u32 progress;         /* current progress of image load */
>>> +       __u32 err_progress;     /* progress at time of error */
>>> +       __u32 err_code;         /* error code */
>>> +};
>> Could this be extended to also collect the image detail?
>>
>>          Image version, name, etc been successfully written to device (flash).
>>
>>          Image version, name, etc is currently running on the device.
>>
>> Or maybe add another query command to do these?
>>
>> So the userland utility will be able to show what image is running and what image is going to run with next cold reboot.
> This status call is intended specifically to show the progress and
> status of the image load. Yilun's comments on patch 5 suggest that
> some of these types of operations could be handled as part of the
> fpga-region class driver
>
> I have patches that have not been submitted yet that contain some
> of these types of features, although not in the class driver (yet).
> Here is a link to documentation for what I have now. Take a look
> at the sysfs node descriptions that are in the control subdirectory.
> These are listed at the bottom of the file.
>
> https://github.com/OPAE/linux-dfl/blob/fpga-ofs-dev/Documentation/ABI/testing/sysfs-driver-intel-m10-bmc-sec-update

Thanks a lot. This is very helpful.

Lizhi

>
> - Russ
>
>
>>
>> Thanks,
>>
>> Lizhi
>>
>>> +
>>> +#define FPGA_IMAGE_LOAD_STATUS _IOR(FPGA_IMAGE_LOAD_MAGIC, 1, struct fpga_image_status)
>>> +
>>>    #endif /* _UAPI_LINUX_FPGA_IMAGE_LOAD_H */
>>> --
>>> 2.25.1
>>>

RE: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Wu, Hao]

> >>>>>> The FPGA Image Load Framework was designed with the concept of
> >>>>>> transferring data to a device without imposing a purpose on the data.
> >>>>>> The expectation is that the lower-level driver or the device will
> >>>>>> validate the data. Is there something fundamentally wrong with that
> >>>>> I think there is something wrong here. As I said before, persistent
> >>>>> storage updating has different software process from some runtime
> >>>>> updating, so the class driver should be aware of what the HW engine
> >>>>> is doing.
> >>>> So far, there are no self-describing images that cause a
> >>>> change in run-time behavior, and I don't think that will
> >>>> happen for the very reason that the class-driver would
> >>>> need to know about it.
> >>> Again, the class driver needs to know what is happening, at some
> >>> abstraction level, to ensure the system is aligned with the HW state.
> >>>
> >>> If the class driver cannot tell the detail, it has to assume the
> >>> whole FPGA region will be changed, and removal & re-enumeration is
> >>> needed.
> >> So we make it a requirement that the self-describing files
> >> cannot make changes that require the class driver to manage
> >> state.
> > The API should not only define what it won't do, but also define what
> > it will do. But the "image load" just specifies the top half of the
> > process. So I don't think this API would be accepted.
> So what is the path forward. It seems like you are saying
> that the self-describing files do not fit in the fpga-mgr.
> Can we reconsider the FPGA Image Load Framework, which does
> not make any assumptions about the contents of the image
> files?

Why we need such "generic data transfer" interface in FPGA
framework? we need to handle the common need for FPGA
devices only, not all devices, like programming FPGA images.
So far we even don't know, what's the hardware response on
these self-describing files, how we define it as a common need
interface in the framework? If you just want to reuse the
fpga-mgr/framework code for your own purpose, Yes, it seems
saving some code for you, but finally it loses flexibility, as it's
not possible to extend common framework for your own
purpose in the future.
 
Thanks
Hao

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/25/21 11:45 PM, Wu, Hao wrote:
>>>>>>>> The FPGA Image Load Framework was designed with the concept of
>>>>>>>> transferring data to a device without imposing a purpose on the data.
>>>>>>>> The expectation is that the lower-level driver or the device will
>>>>>>>> validate the data. Is there something fundamentally wrong with that
>>>>>>> I think there is something wrong here. As I said before, persistent
>>>>>>> storage updating has different software process from some runtime
>>>>>>> updating, so the class driver should be aware of what the HW engine
>>>>>>> is doing.
>>>>>> So far, there are no self-describing images that cause a
>>>>>> change in run-time behavior, and I don't think that will
>>>>>> happen for the very reason that the class-driver would
>>>>>> need to know about it.
>>>>> Again, the class driver needs to know what is happening, at some
>>>>> abstraction level, to ensure the system is aligned with the HW state.
>>>>>
>>>>> If the class driver cannot tell the detail, it has to assume the
>>>>> whole FPGA region will be changed, and removal & re-enumeration is
>>>>> needed.
>>>> So we make it a requirement that the self-describing files
>>>> cannot make changes that require the class driver to manage
>>>> state.
>>> The API should not only define what it won't do, but also define what
>>> it will do. But the "image load" just specifies the top half of the
>>> process. So I don't think this API would be accepted.
>> So what is the path forward. It seems like you are saying
>> that the self-describing files do not fit in the fpga-mgr.
>> Can we reconsider the FPGA Image Load Framework, which does
>> not make any assumptions about the contents of the image
>> files?
> Why we need such "generic data transfer" interface in FPGA
> framework? 
Are you referring to the use of self-describing files?
or the generic nature of this class driver?
> we need to handle the common need for FPGA
> devices only, not all devices, like programming FPGA images.
> So far we even don't know, what's the hardware response on
> these self-describing files, how we define it as a common need
> interface in the framework?
The class driver does not _need_ to reside in the FPGA
framework. I sent an inquiry to the maintainer of the
Firmware update subsystem (and cc'd the kernel mailing list)
and received no responses. I placed it under the FPGA
framework only because the first user of the class driver
is an FPGA driver.

> If you just want to reuse the
> fpga-mgr/framework code for your own purpose, Yes, it seems
> saving some code for you, but finally it loses flexibility, as it's
> not possible to extend common framework for your own
> purpose in the future.
If I understand correctly, you are saying that it doesn't
fit well in the FPGA manager, because not all file types
fit the definition of a firmware update? And future file
types may not fit in fpga-mgr context?

- Russ
>  
> Thanks
> Hao

RE: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Wu, Hao]

> >>> The API should not only define what it won't do, but also define what
> >>> it will do. But the "image load" just specifies the top half of the
> >>> process. So I don't think this API would be accepted.
> >> So what is the path forward. It seems like you are saying
> >> that the self-describing files do not fit in the fpga-mgr.
> >> Can we reconsider the FPGA Image Load Framework, which does
> >> not make any assumptions about the contents of the image
> >> files?
> > Why we need such "generic data transfer" interface in FPGA
> > framework?
> Are you referring to the use of self-describing files?
> or the generic nature of this class driver?

Yes, why this is under FPGA framework? Per your description that
it can be used to transfer any data, e.g. BMC images, some device
specific data (self-describing image?). Let's take this as example,
if FPGA device is replaced with ASIC on N3000, do you still want
to use FPGA image load framework to transfer your device specific
data, e.g. BMC images? I really hope that FPGA framework code only
focus on common usage of FPGA. 

> > we need to handle the common need for FPGA
> > devices only, not all devices, like programming FPGA images.
> > So far we even don't know, what's the hardware response on
> > these self-describing files, how we define it as a common need
> > interface in the framework?
> The class driver does not _need_ to reside in the FPGA
> framework. I sent an inquiry to the maintainer of the
> Firmware update subsystem (and cc'd the kernel mailing list)
> and received no responses. I placed it under the FPGA
> framework only because the first user of the class driver
> is an FPGA driver.
You must have enough justifications why this needs to be included
for everybody not for our own case.

> 
> > If you just want to reuse the
> > fpga-mgr/framework code for your own purpose, Yes, it seems
> > saving some code for you, but finally it loses flexibility, as it's
> > not possible to extend common framework for your own
> > purpose in the future.
> If I understand correctly, you are saying that it doesn't
> fit well in the FPGA manager, because not all file types
> fit the definition of a firmware update? And future file
> types may not fit in fpga-mgr context?

Let's split the use cases, I think the use case that update a persistent
storage for FPGA image, and later use hardware logic (FPGA loader)
to load it into FPGA. This sounds like a common usage for FPGA
devices, so I think this is why Yilun propose to have this part to be
covered by fpga-mgr. But for other cases in your description, e.g.
BMC images, device specific data, self-describing image and etc,
they are out of scope of FPGA.

Actually I don't fully understand why we need to introduce the
"self-describing image" as a common data transfer interface, if
I remember correctly, for N3000, different sub drivers will own
different hardware sub function blocks, why expose such a new
shared communication channel? If "self-describing image" is a
request to one of the sub function block, why not just expose
new interface in such hardware block per modularization? I
have some concern that this new requirement may break
current driver architecture for N3000.

Hao

> 
> - Russ
> >
> > Thanks
> > Hao

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Russ Weight]

On 10/26/21 8:29 PM, Wu, Hao wrote:
>>>>> The API should not only define what it won't do, but also define what
>>>>> it will do. But the "image load" just specifies the top half of the
>>>>> process. So I don't think this API would be accepted.
>>>> So what is the path forward. It seems like you are saying
>>>> that the self-describing files do not fit in the fpga-mgr.
>>>> Can we reconsider the FPGA Image Load Framework, which does
>>>> not make any assumptions about the contents of the image
>>>> files?
>>> Why we need such "generic data transfer" interface in FPGA
>>> framework?
>> Are you referring to the use of self-describing files?
>> or the generic nature of this class driver?
> Yes, why this is under FPGA framework? Per your description that
> it can be used to transfer any data, e.g. BMC images, some device
> specific data (self-describing image?). Let's take this as example,
> if FPGA device is replaced with ASIC on N3000, do you still want
> to use FPGA image load framework to transfer your device specific
> data, e.g. BMC images? I really hope that FPGA framework code only
> focus on common usage of FPGA. 
>
>>> we need to handle the common need for FPGA
>>> devices only, not all devices, like programming FPGA images.
>>> So far we even don't know, what's the hardware response on
>>> these self-describing files, how we define it as a common need
>>> interface in the framework?
>> The class driver does not _need_ to reside in the FPGA
>> framework. I sent an inquiry to the maintainer of the
>> Firmware update subsystem (and cc'd the kernel mailing list)
>> and received no responses. I placed it under the FPGA
>> framework only because the first user of the class driver
>> is an FPGA driver.
> You must have enough justifications why this needs to be included
> for everybody not for our own case.

How do we justify it when there are currently no other known
users? I can go ahead and work up some patches for the firmware
subsystem, if we can resolve the other concerns below.

>>> If you just want to reuse the
>>> fpga-mgr/framework code for your own purpose, Yes, it seems
>>> saving some code for you, but finally it loses flexibility, as it's
>>> not possible to extend common framework for your own
>>> purpose in the future.
>> If I understand correctly, you are saying that it doesn't
>> fit well in the FPGA manager, because not all file types
>> fit the definition of a firmware update? And future file
>> types may not fit in fpga-mgr context?
> Let's split the use cases, I think the use case that update a persistent
> storage for FPGA image, and later use hardware logic (FPGA loader)
> to load it into FPGA. This sounds like a common usage for FPGA
> devices, so I think this is why Yilun propose to have this part to be
> covered by fpga-mgr. But for other cases in your description, e.g.
> BMC images, device specific data, self-describing image and etc,
> they are out of scope of FPGA.

Self-describing files are not something new to us; _ALL_ of the image
files that we send to our FPGA cards, including the N3000 FPGA and BMC
images, root-entry hashes, key cancellations, etc. are self-describing
files. They always have been.


>
> Actually I don't fully understand why we need to introduce the
> "self-describing image" as a common data transfer interface, if
> I remember correctly, for N3000, different sub drivers will own
> different hardware sub function blocks, why expose such a new
> shared communication channel?

There is no change here. The N3000 files are self describing. The
secure-update sub-driver of the MAX10 BMC invokes the class driver,
funnels image data to the BMC, performs handshakes with the BMC,
and ultimately returns status through the class driver. All images
that are sent to the FPGA card follow this same path - and it works
fine.

To try to split out the purposes of each self-describing file to
use different kernel APIs means interfacing multiple class drivers
to the same MAX10 sub-driver. I think it also means replicating
code.

- Russ
> If "self-describing image" is a
> request to one of the sub function block, why not just expose
> new interface in such hardware block per modularization? I
> have some concern that this new requirement may break
> current driver architecture for N3000.
>
> Hao
>
>> - Russ
>>> Thanks
>>> Hao

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Tom Rix]

On 10/27/21 8:11 AM, Russ Weight wrote:
>
> On 10/26/21 8:29 PM, Wu, Hao wrote:
>>>>>> The API should not only define what it won't do, but also define what
>>>>>> it will do. But the "image load" just specifies the top half of the
>>>>>> process. So I don't think this API would be accepted.
>>>>> So what is the path forward. It seems like you are saying
>>>>> that the self-describing files do not fit in the fpga-mgr.
>>>>> Can we reconsider the FPGA Image Load Framework, which does
>>>>> not make any assumptions about the contents of the image
>>>>> files?
>>>> Why we need such "generic data transfer" interface in FPGA
>>>> framework?
>>> Are you referring to the use of self-describing files?
>>> or the generic nature of this class driver?
>> Yes, why this is under FPGA framework? Per your description that
>> it can be used to transfer any data, e.g. BMC images, some device
>> specific data (self-describing image?). Let's take this as example,
>> if FPGA device is replaced with ASIC on N3000, do you still want
>> to use FPGA image load framework to transfer your device specific
>> data, e.g. BMC images? I really hope that FPGA framework code only
>> focus on common usage of FPGA.
>>
>>>> we need to handle the common need for FPGA
>>>> devices only, not all devices, like programming FPGA images.
>>>> So far we even don't know, what's the hardware response on
>>>> these self-describing files, how we define it as a common need
>>>> interface in the framework?
>>> The class driver does not _need_ to reside in the FPGA
>>> framework. I sent an inquiry to the maintainer of the
>>> Firmware update subsystem (and cc'd the kernel mailing list)
>>> and received no responses. I placed it under the FPGA
>>> framework only because the first user of the class driver
>>> is an FPGA driver.
>> You must have enough justifications why this needs to be included
>> for everybody not for our own case.
> How do we justify it when there are currently no other known
> users? I can go ahead and work up some patches for the firmware
> subsystem, if we can resolve the other concerns below.
>
>>>> If you just want to reuse the
>>>> fpga-mgr/framework code for your own purpose, Yes, it seems
>>>> saving some code for you, but finally it loses flexibility, as it's
>>>> not possible to extend common framework for your own
>>>> purpose in the future.
>>> If I understand correctly, you are saying that it doesn't
>>> fit well in the FPGA manager, because not all file types
>>> fit the definition of a firmware update? And future file
>>> types may not fit in fpga-mgr context?
>> Let's split the use cases, I think the use case that update a persistent
>> storage for FPGA image, and later use hardware logic (FPGA loader)
>> to load it into FPGA. This sounds like a common usage for FPGA
>> devices, so I think this is why Yilun propose to have this part to be
>> covered by fpga-mgr. But for other cases in your description, e.g.
>> BMC images, device specific data, self-describing image and etc,
>> they are out of scope of FPGA.
> Self-describing files are not something new to us; _ALL_ of the image
> files that we send to our FPGA cards, including the N3000 FPGA and BMC
> images, root-entry hashes, key cancellations, etc. are self-describing
> files. They always have been.
>
>    
>> Actually I don't fully understand why we need to introduce the
>> "self-describing image" as a common data transfer interface, if
>> I remember correctly, for N3000, different sub drivers will own
>> different hardware sub function blocks, why expose such a new
>> shared communication channel?
> There is no change here. The N3000 files are self describing. The
> secure-update sub-driver of the MAX10 BMC invokes the class driver,
> funnels image data to the BMC, performs handshakes with the BMC,
> and ultimately returns status through the class driver. All images
> that are sent to the FPGA card follow this same path - and it works
> fine.
>
> To try to split out the purposes of each self-describing file to
> use different kernel APIs means interfacing multiple class drivers
> to the same MAX10 sub-driver. I think it also means replicating
> code.

Could the split be ?

add max10 bits mfd/

move image updating out of the kernel and into an uio driver

Tom

>
> - Russ
>> If "self-describing image" is a
>> request to one of the sub function block, why not just expose
>> new interface in such hardware block per modularization? I
>> have some concern that this new requirement may break
>> current driver architecture for N3000.
>>
>> Hao
>>
>>> - Russ
>>>> Thanks
>>>> Hao

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Wed, Oct 27, 2021 at 08:34:16AM -0700, Tom Rix wrote:
> 
> On 10/27/21 8:11 AM, Russ Weight wrote:
> > 
> > On 10/26/21 8:29 PM, Wu, Hao wrote:
> > > > > > > The API should not only define what it won't do, but also define what
> > > > > > > it will do. But the "image load" just specifies the top half of the
> > > > > > > process. So I don't think this API would be accepted.
> > > > > > So what is the path forward. It seems like you are saying
> > > > > > that the self-describing files do not fit in the fpga-mgr.
> > > > > > Can we reconsider the FPGA Image Load Framework, which does
> > > > > > not make any assumptions about the contents of the image
> > > > > > files?
> > > > > Why we need such "generic data transfer" interface in FPGA
> > > > > framework?
> > > > Are you referring to the use of self-describing files?
> > > > or the generic nature of this class driver?
> > > Yes, why this is under FPGA framework? Per your description that
> > > it can be used to transfer any data, e.g. BMC images, some device
> > > specific data (self-describing image?). Let's take this as example,
> > > if FPGA device is replaced with ASIC on N3000, do you still want
> > > to use FPGA image load framework to transfer your device specific
> > > data, e.g. BMC images? I really hope that FPGA framework code only
> > > focus on common usage of FPGA.
> > > 
> > > > > we need to handle the common need for FPGA
> > > > > devices only, not all devices, like programming FPGA images.
> > > > > So far we even don't know, what's the hardware response on
> > > > > these self-describing files, how we define it as a common need
> > > > > interface in the framework?
> > > > The class driver does not _need_ to reside in the FPGA
> > > > framework. I sent an inquiry to the maintainer of the
> > > > Firmware update subsystem (and cc'd the kernel mailing list)
> > > > and received no responses. I placed it under the FPGA
> > > > framework only because the first user of the class driver
> > > > is an FPGA driver.
> > > You must have enough justifications why this needs to be included
> > > for everybody not for our own case.
> > How do we justify it when there are currently no other known
> > users? I can go ahead and work up some patches for the firmware
> > subsystem, if we can resolve the other concerns below.
> > 
> > > > > If you just want to reuse the
> > > > > fpga-mgr/framework code for your own purpose, Yes, it seems
> > > > > saving some code for you, but finally it loses flexibility, as it's
> > > > > not possible to extend common framework for your own
> > > > > purpose in the future.
> > > > If I understand correctly, you are saying that it doesn't
> > > > fit well in the FPGA manager, because not all file types
> > > > fit the definition of a firmware update? And future file
> > > > types may not fit in fpga-mgr context?
> > > Let's split the use cases, I think the use case that update a persistent
> > > storage for FPGA image, and later use hardware logic (FPGA loader)
> > > to load it into FPGA. This sounds like a common usage for FPGA
> > > devices, so I think this is why Yilun propose to have this part to be
> > > covered by fpga-mgr. But for other cases in your description, e.g.
> > > BMC images, device specific data, self-describing image and etc,
> > > they are out of scope of FPGA.
> > Self-describing files are not something new to us; _ALL_ of the image
> > files that we send to our FPGA cards, including the N3000 FPGA and BMC
> > images, root-entry hashes, key cancellations, etc. are self-describing
> > files. They always have been.
> > 
> > > Actually I don't fully understand why we need to introduce the
> > > "self-describing image" as a common data transfer interface, if
> > > I remember correctly, for N3000, different sub drivers will own
> > > different hardware sub function blocks, why expose such a new
> > > shared communication channel?
> > There is no change here. The N3000 files are self describing. The
> > secure-update sub-driver of the MAX10 BMC invokes the class driver,
> > funnels image data to the BMC, performs handshakes with the BMC,
> > and ultimately returns status through the class driver. All images
> > that are sent to the FPGA card follow this same path - and it works
> > fine.
> > 
> > To try to split out the purposes of each self-describing file to
> > use different kernel APIs means interfacing multiple class drivers
> > to the same MAX10 sub-driver. I think it also means replicating
> > code.
> 
> Could the split be ?
> 
> add max10 bits mfd/
> 
> move image updating out of the kernel and into an uio driver

I'm afraid an uio driver doesn't help in this case. The image updating
is not an independent device, it may dynamically change other hardwares.
So it is better the image updating driver works as an low level driver
which provides services to other feature drivers.

Thanks,
Yilun

> 
> Tom
> 
> > 
> > - Russ
> > > If "self-describing image" is a
> > > request to one of the sub function block, why not just expose
> > > new interface in such hardware block per modularization? I
> > > have some concern that this new requirement may break
> > > current driver architecture for N3000.
> > > 
> > > Hao
> > > 
> > > > - Russ
> > > > > Thanks
> > > > > Hao

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Tom Rix]

On 10/28/21 8:09 AM, Xu Yilun wrote:
> On Wed, Oct 27, 2021 at 08:34:16AM -0700, Tom Rix wrote:
>> On 10/27/21 8:11 AM, Russ Weight wrote:
>>> On 10/26/21 8:29 PM, Wu, Hao wrote:
>>>>>>>> The API should not only define what it won't do, but also define what
>>>>>>>> it will do. But the "image load" just specifies the top half of the
>>>>>>>> process. So I don't think this API would be accepted.
>>>>>>> So what is the path forward. It seems like you are saying
>>>>>>> that the self-describing files do not fit in the fpga-mgr.
>>>>>>> Can we reconsider the FPGA Image Load Framework, which does
>>>>>>> not make any assumptions about the contents of the image
>>>>>>> files?
>>>>>> Why we need such "generic data transfer" interface in FPGA
>>>>>> framework?
>>>>> Are you referring to the use of self-describing files?
>>>>> or the generic nature of this class driver?
>>>> Yes, why this is under FPGA framework? Per your description that
>>>> it can be used to transfer any data, e.g. BMC images, some device
>>>> specific data (self-describing image?). Let's take this as example,
>>>> if FPGA device is replaced with ASIC on N3000, do you still want
>>>> to use FPGA image load framework to transfer your device specific
>>>> data, e.g. BMC images? I really hope that FPGA framework code only
>>>> focus on common usage of FPGA.
>>>>
>>>>>> we need to handle the common need for FPGA
>>>>>> devices only, not all devices, like programming FPGA images.
>>>>>> So far we even don't know, what's the hardware response on
>>>>>> these self-describing files, how we define it as a common need
>>>>>> interface in the framework?
>>>>> The class driver does not _need_ to reside in the FPGA
>>>>> framework. I sent an inquiry to the maintainer of the
>>>>> Firmware update subsystem (and cc'd the kernel mailing list)
>>>>> and received no responses. I placed it under the FPGA
>>>>> framework only because the first user of the class driver
>>>>> is an FPGA driver.
>>>> You must have enough justifications why this needs to be included
>>>> for everybody not for our own case.
>>> How do we justify it when there are currently no other known
>>> users? I can go ahead and work up some patches for the firmware
>>> subsystem, if we can resolve the other concerns below.
>>>
>>>>>> If you just want to reuse the
>>>>>> fpga-mgr/framework code for your own purpose, Yes, it seems
>>>>>> saving some code for you, but finally it loses flexibility, as it's
>>>>>> not possible to extend common framework for your own
>>>>>> purpose in the future.
>>>>> If I understand correctly, you are saying that it doesn't
>>>>> fit well in the FPGA manager, because not all file types
>>>>> fit the definition of a firmware update? And future file
>>>>> types may not fit in fpga-mgr context?
>>>> Let's split the use cases, I think the use case that update a persistent
>>>> storage for FPGA image, and later use hardware logic (FPGA loader)
>>>> to load it into FPGA. This sounds like a common usage for FPGA
>>>> devices, so I think this is why Yilun propose to have this part to be
>>>> covered by fpga-mgr. But for other cases in your description, e.g.
>>>> BMC images, device specific data, self-describing image and etc,
>>>> they are out of scope of FPGA.
>>> Self-describing files are not something new to us; _ALL_ of the image
>>> files that we send to our FPGA cards, including the N3000 FPGA and BMC
>>> images, root-entry hashes, key cancellations, etc. are self-describing
>>> files. They always have been.
>>>
>>>> Actually I don't fully understand why we need to introduce the
>>>> "self-describing image" as a common data transfer interface, if
>>>> I remember correctly, for N3000, different sub drivers will own
>>>> different hardware sub function blocks, why expose such a new
>>>> shared communication channel?
>>> There is no change here. The N3000 files are self describing. The
>>> secure-update sub-driver of the MAX10 BMC invokes the class driver,
>>> funnels image data to the BMC, performs handshakes with the BMC,
>>> and ultimately returns status through the class driver. All images
>>> that are sent to the FPGA card follow this same path - and it works
>>> fine.
>>>
>>> To try to split out the purposes of each self-describing file to
>>> use different kernel APIs means interfacing multiple class drivers
>>> to the same MAX10 sub-driver. I think it also means replicating
>>> code.
>> Could the split be ?
>>
>> add max10 bits mfd/
>>
>> move image updating out of the kernel and into an uio driver
> I'm afraid an uio driver doesn't help in this case. The image updating
> is not an independent device, it may dynamically change other hardwares.
> So it is better the image updating driver works as an low level driver
> which provides services to other feature drivers.

Ok.

Since this is dfl specific could a 'write' op be added to fme_fops ?

Tom

>
> Thanks,
> Yilun
>
>> Tom
>>
>>> - Russ
>>>> If "self-describing image" is a
>>>> request to one of the sub function block, why not just expose
>>>> new interface in such hardware block per modularization? I
>>>> have some concern that this new requirement may break
>>>> current driver architecture for N3000.
>>>>
>>>> Hao
>>>>
>>>>> - Russ
>>>>>> Thanks
>>>>>> Hao

Re: [PATCH v17 0/5] FPGA Image Load (previously Security Manager)

[Xu Yilun]

On Thu, Oct 28, 2021 at 09:08:03AM -0700, Tom Rix wrote:
> 
> On 10/28/21 8:09 AM, Xu Yilun wrote:
> > On Wed, Oct 27, 2021 at 08:34:16AM -0700, Tom Rix wrote:
> > > On 10/27/21 8:11 AM, Russ Weight wrote:
> > > > On 10/26/21 8:29 PM, Wu, Hao wrote:
> > > > > > > > > The API should not only define what it won't do, but also define what
> > > > > > > > > it will do. But the "image load" just specifies the top half of the
> > > > > > > > > process. So I don't think this API would be accepted.
> > > > > > > > So what is the path forward. It seems like you are saying
> > > > > > > > that the self-describing files do not fit in the fpga-mgr.
> > > > > > > > Can we reconsider the FPGA Image Load Framework, which does
> > > > > > > > not make any assumptions about the contents of the image
> > > > > > > > files?
> > > > > > > Why we need such "generic data transfer" interface in FPGA
> > > > > > > framework?
> > > > > > Are you referring to the use of self-describing files?
> > > > > > or the generic nature of this class driver?
> > > > > Yes, why this is under FPGA framework? Per your description that
> > > > > it can be used to transfer any data, e.g. BMC images, some device
> > > > > specific data (self-describing image?). Let's take this as example,
> > > > > if FPGA device is replaced with ASIC on N3000, do you still want
> > > > > to use FPGA image load framework to transfer your device specific
> > > > > data, e.g. BMC images? I really hope that FPGA framework code only
> > > > > focus on common usage of FPGA.
> > > > > 
> > > > > > > we need to handle the common need for FPGA
> > > > > > > devices only, not all devices, like programming FPGA images.
> > > > > > > So far we even don't know, what's the hardware response on
> > > > > > > these self-describing files, how we define it as a common need
> > > > > > > interface in the framework?
> > > > > > The class driver does not _need_ to reside in the FPGA
> > > > > > framework. I sent an inquiry to the maintainer of the
> > > > > > Firmware update subsystem (and cc'd the kernel mailing list)
> > > > > > and received no responses. I placed it under the FPGA
> > > > > > framework only because the first user of the class driver
> > > > > > is an FPGA driver.
> > > > > You must have enough justifications why this needs to be included
> > > > > for everybody not for our own case.
> > > > How do we justify it when there are currently no other known
> > > > users? I can go ahead and work up some patches for the firmware
> > > > subsystem, if we can resolve the other concerns below.
> > > > 
> > > > > > > If you just want to reuse the
> > > > > > > fpga-mgr/framework code for your own purpose, Yes, it seems
> > > > > > > saving some code for you, but finally it loses flexibility, as it's
> > > > > > > not possible to extend common framework for your own
> > > > > > > purpose in the future.
> > > > > > If I understand correctly, you are saying that it doesn't
> > > > > > fit well in the FPGA manager, because not all file types
> > > > > > fit the definition of a firmware update? And future file
> > > > > > types may not fit in fpga-mgr context?
> > > > > Let's split the use cases, I think the use case that update a persistent
> > > > > storage for FPGA image, and later use hardware logic (FPGA loader)
> > > > > to load it into FPGA. This sounds like a common usage for FPGA
> > > > > devices, so I think this is why Yilun propose to have this part to be
> > > > > covered by fpga-mgr. But for other cases in your description, e.g.
> > > > > BMC images, device specific data, self-describing image and etc,
> > > > > they are out of scope of FPGA.
> > > > Self-describing files are not something new to us; _ALL_ of the image
> > > > files that we send to our FPGA cards, including the N3000 FPGA and BMC
> > > > images, root-entry hashes, key cancellations, etc. are self-describing
> > > > files. They always have been.
> > > > 
> > > > > Actually I don't fully understand why we need to introduce the
> > > > > "self-describing image" as a common data transfer interface, if
> > > > > I remember correctly, for N3000, different sub drivers will own
> > > > > different hardware sub function blocks, why expose such a new
> > > > > shared communication channel?
> > > > There is no change here. The N3000 files are self describing. The
> > > > secure-update sub-driver of the MAX10 BMC invokes the class driver,
> > > > funnels image data to the BMC, performs handshakes with the BMC,
> > > > and ultimately returns status through the class driver. All images
> > > > that are sent to the FPGA card follow this same path - and it works
> > > > fine.
> > > > 
> > > > To try to split out the purposes of each self-describing file to
> > > > use different kernel APIs means interfacing multiple class drivers
> > > > to the same MAX10 sub-driver. I think it also means replicating
> > > > code.
> > > Could the split be ?
> > > 
> > > add max10 bits mfd/
> > > 
> > > move image updating out of the kernel and into an uio driver
> > I'm afraid an uio driver doesn't help in this case. The image updating
> > is not an independent device, it may dynamically change other hardwares.
> > So it is better the image updating driver works as an low level driver
> > which provides services to other feature drivers.
> 
> Ok.
> 
> Since this is dfl specific could a 'write' op be added to fme_fops ?

We still need to have a clear definition or classification of what the
changes could be so that we discuss which operations could be added to
which driver. Or we have to assume the FME is changed and has to be
removed before loading and re-enumerated afterward. That seems not what
everyone want.

Thanks,
Yilun

> 
> Tom
> 
> > 
> > Thanks,
> > Yilun
> > 
> > > Tom
> > > 
> > > > - Russ
> > > > > If "self-describing image" is a
> > > > > request to one of the sub function block, why not just expose
> > > > > new interface in such hardware block per modularization? I
> > > > > have some concern that this new requirement may break
> > > > > current driver architecture for N3000.
> > > > > 
> > > > > Hao
> > > > > 
> > > > > > - Russ
> > > > > > > Thanks
> > > > > > > Hao