From: Tom Rix <trix@redhat.com> To: Lizhi Hou <lizhi.hou@xilinx.com>, linux-kernel@vger.kernel.org, "Wu, Hao" <hao.wu@intel.com>, "mdf@kernel.org" <mdf@kernel.org> Cc: linux-fpga@vger.kernel.org, maxz@xilinx.com, sonal.santan@xilinx.com, yliu@xilinx.com, michal.simek@xilinx.com, stefanos@xilinx.com, devicetree@vger.kernel.org, mdf@kernel.org, robh@kernel.org Subject: RFC : [PATCH V7 XRT Alveo 00/20] XRT Alveo driver overview Date: Thu, 3 Jun 2021 07:23:10 -0700 [thread overview] Message-ID: <68e85a4f-4a10-1ff9-0443-aa565878c855@redhat.com> (raw) In-Reply-To: <bdd31ecc-f6c5-392b-1609-f05efcba7f88@xilinx.com> On 6/2/21 3:34 PM, Lizhi Hou wrote: > Hi Moritz and Tom, > > Thanks a lot for reviewing the XRT patches. This v7 patches contains > our latest fixes for all review comments. And it seems auto build > robot is happy with it as well. Do you have more comment against v7? > What will be the next step for XRT patches if it looks ok to you? Please do not add any features to the patchset. When you add a feature, that tends to reset the review process. For little stuff. I need to look at the documentation again, iirc that was the only issue in v6. For big stuff. At the beginning some possible refactor questions were raised, I don't know if they have been resolved. ex/ some of the leaf drivers like clock should be moved the clock subsystem the xrt/ dir vs existing flat, we don't want both. is your xrt bus consistent with dfl bus, can it support adding devices outside of fpga/xrt how clean are the public headers Since flipflopping on refactoring is a painful, let's try to get a consensus before moving forward. This may mean the patchset is committed by parts as it and the existing code changes. Lets start with the directory structure. Since there are a lot of files in xrt, the xrt/ makes sense. For other fpga's with a lot of files, say > 3, we need to change ex/ dfl-foo.c to dfl/foo.c to be consistent. The Kconfig and Makefiles for xrt/ and dfl/ would only be in that top subdir. Which i think is a change for xrt/ I'll whip up something for dfl/ to move things along. Tom > > Thanks, > > Lizhi > > > On 05/27/2021 05:49 PM, Lizhi Hou wrote: >> Hello, >> >> This is V7 of patch series which adds management physical function >> driver >> for Xilinx Alveo PCIe accelerator cards. >> https://www.xilinx.com/products/boards-and-kits/alveo.html >> >> This driver is part of Xilinx Runtime (XRT) open source stack. >> >> XILINX ALVEO PLATFORM ARCHITECTURE >> >> Alveo PCIe FPGA based platforms have a static *shell* partition and a >> partial re-configurable *user* partition. The shell partition is >> automatically loaded from flash when host is booted and PCIe is >> enumerated >> by BIOS. Shell cannot be changed till the next cold reboot. The shell >> exposes two PCIe physical functions: >> >> 1. management physical function >> 2. user physical function >> >> The patch series includes Documentation/xrt.rst which describes Alveo >> platform, XRT driver architecture and deployment model in more detail. >> >> Users compile their high level design in C/C++/OpenCL or RTL into FPGA >> image using Vitis tools. >> https://www.xilinx.com/products/design-tools/vitis/vitis-platform.html >> >> The compiled image is packaged as xclbin which contains partial >> bitstream >> for the user partition and necessary metadata. Users can dynamically >> swap >> the image running on the user partition in order to switch between >> different workloads by loading different xclbins. >> >> XRT DRIVERS FOR XILINX ALVEO >> >> XRT Linux kernel driver *xrt-mgmt* binds to management physical >> function of >> Alveo platform. The modular driver framework is organized into several >> platform drivers which primarily handle the following functionality: >> >> 1. Loading firmware container also called xsabin at driver attach time >> 2. Loading of user compiled xclbin with FPGA Manager integration >> 3. Clock scaling of image running on user partition >> 4. In-band sensors: temp, voltage, power, etc. >> 5. Device reset and rescan >> >> The platform drivers are packaged into *xrt-lib* helper module with well >> defined interfaces. The module provides a pseudo-bus implementation >> for the >> platform drivers. More details on the driver model can be found in >> Documentation/xrt.rst. >> >> User physical function driver is not included in this patch series. >> >> LIBFDT REQUIREMENT >> >> XRT driver infrastructure uses Device Tree as a metadata format to >> discover >> HW subsystems in the Alveo PCIe device. The Device Tree schema used >> by XRT >> is documented in Documentation/xrt.rst. >> >> TESTING AND VALIDATION >> >> xrt-mgmt driver can be tested with full XRT open source stack which >> includes user space libraries, board utilities and (out of tree) first >> generation user physical function driver xocl. XRT open source runtime >> stack is available at https://github.com/Xilinx/XRT >> >> Complete documentation for XRT open source stack including sections on >> Alveo/XRT security and platform architecture can be found here: >> >> https://xilinx.github.io/XRT/master/html/index.html >> https://xilinx.github.io/XRT/master/html/security.html >> https://xilinx.github.io/XRT/master/html/platforms_partitions.html >> >> Changes since v6: >> - Resolved grammatical errors and cleaned up taxonomy in xrt.rst >> documentation. >> - Fixed clang warnings. >> - Updated code base to include v6 code review comments. >> >> Changes since v5: >> - Revert all changes 'mgnt/MGNT' back to 'mgmt/MGMT' >> - Updated code base to include v5 code review comments. >> xrt.rst: address grammar and taxonomy >> subdev_id.h: defines XRT_SUBDEV_INVALID = 0 >> xclbin.c: change shift operation to be_to_cpu >> - Resolved kernel test robot errors. >> >> Changes since v4: >> - Added xrt_bus_type and xrt_device. All sub devices were changed from >> platform_bus_type/platform_device to xrt_bus_type/xrt_device. >> - Renamed xrt-mgmt driver to xrt-mgnt driver. >> - Replaced 'MGMT' with 'MGNT' and 'mgmt' with 'mgnt' in code and file >> names >> - Moved pci function calls from infrastructure to xrt-mgnt driver. >> - Renamed files: mgmt/main.c -> mgnt/xmgnt-main.c >> mgmt/main-region.c -> mgnt/xmgnt-main-region.c >> include/xmgmt-main.h -> include/xmgnt-main.h >> mgmt/fmgr-drv.c -> mgnt/xrt-mgr.c >> mgmt/fmgr.h -> mgnt/xrt-mgr.h >> - Updated code base to include v4 code review comments. >> >> Changes since v3: >> - Leaf drivers use regmap-mmio to access hardware registers. >> - Renamed driver module: xmgmt.ko -> xrt-mgmt.ko >> - Renamed files: calib.[c|h] -> ddr_calibration.[c|h], >> lib/main.[c|h] -> lib/lib-drv.[c|h], >> mgmt/main-impl.h - > mgmt/xmgnt.h >> - Updated code base to include v3 code review comments. >> >> Changes since v2: >> - Streamlined the driver framework into *xleaf*, *group* and *xroot* >> - Updated documentation to show the driver model with examples >> - Addressed kernel test robot errors >> - Added a selftest for basic driver framework >> - Documented device tree schema >> - Removed need to export libfdt symbols >> >> Changes since v1: >> - Updated the driver to use fpga_region and fpga_bridge for FPGA >> programming >> - Dropped platform drivers not related to PR programming to focus on XRT >> core framework >> - Updated Documentation/fpga/xrt.rst with information on XRT core >> framework >> - Addressed checkpatch issues >> - Dropped xrt- prefix from some header files >> >> For reference V6 version of patch series can be found here: >> >> https://lore.kernel.org/lkml/20210512015339.5649-1-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-2-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-3-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-4-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-5-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-6-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-7-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-8-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-9-lizhi.hou@xilinx.com/ >> https://lore.kernel.org/lkml/20210512015339.5649-10-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-11-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-12-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-13-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-14-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-15-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-16-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-17-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-18-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-19-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-20-lizhi.hou@xilinx.com/ >> >> https://lore.kernel.org/lkml/20210512015339.5649-21-lizhi.hou@xilinx.com/ >> >> >> Lizhi Hou (20): >> Documentation: fpga: Add a document describing XRT Alveo drivers >> fpga: xrt: driver metadata helper functions >> fpga: xrt: xclbin file helper functions >> fpga: xrt: xrt-lib driver manager >> fpga: xrt: group driver >> fpga: xrt: char dev node helper functions >> fpga: xrt: root driver infrastructure >> fpga: xrt: driver infrastructure >> fpga: xrt: management physical function driver (root) >> fpga: xrt: main driver for management function device >> fpga: xrt: fpga-mgr and region implementation for xclbin download >> fpga: xrt: VSEC driver >> fpga: xrt: User Clock Subsystem driver >> fpga: xrt: ICAP driver >> fpga: xrt: devctl xrt driver >> fpga: xrt: clock driver >> fpga: xrt: clock frequency counter driver >> fpga: xrt: DDR calibration driver >> fpga: xrt: partition isolation driver >> fpga: xrt: Kconfig and Makefile updates for XRT drivers >> >> Documentation/fpga/index.rst | 1 + >> Documentation/fpga/xrt.rst | 870 ++++++++++++++++++ >> MAINTAINERS | 11 + >> drivers/Makefile | 1 + >> drivers/fpga/Kconfig | 2 + >> drivers/fpga/Makefile | 5 + >> drivers/fpga/xrt/Kconfig | 8 + >> drivers/fpga/xrt/include/events.h | 45 + >> drivers/fpga/xrt/include/group.h | 25 + >> drivers/fpga/xrt/include/metadata.h | 236 +++++ >> drivers/fpga/xrt/include/subdev_id.h | 39 + >> drivers/fpga/xrt/include/xclbin-helper.h | 48 + >> drivers/fpga/xrt/include/xdevice.h | 131 +++ >> drivers/fpga/xrt/include/xleaf.h | 205 +++++ >> drivers/fpga/xrt/include/xleaf/axigate.h | 23 + >> drivers/fpga/xrt/include/xleaf/clkfreq.h | 21 + >> drivers/fpga/xrt/include/xleaf/clock.h | 29 + >> .../fpga/xrt/include/xleaf/ddr_calibration.h | 28 + >> drivers/fpga/xrt/include/xleaf/devctl.h | 40 + >> drivers/fpga/xrt/include/xleaf/icap.h | 27 + >> drivers/fpga/xrt/include/xmgmt-main.h | 34 + >> drivers/fpga/xrt/include/xroot.h | 117 +++ >> drivers/fpga/xrt/lib/Kconfig | 17 + >> drivers/fpga/xrt/lib/Makefile | 30 + >> drivers/fpga/xrt/lib/cdev.c | 209 +++++ >> drivers/fpga/xrt/lib/group.c | 278 ++++++ >> drivers/fpga/xrt/lib/lib-drv.c | 328 +++++++ >> drivers/fpga/xrt/lib/lib-drv.h | 21 + >> drivers/fpga/xrt/lib/subdev.c | 859 +++++++++++++++++ >> drivers/fpga/xrt/lib/subdev_pool.h | 53 ++ >> drivers/fpga/xrt/lib/xclbin.c | 381 ++++++++ >> drivers/fpga/xrt/lib/xleaf/axigate.c | 325 +++++++ >> drivers/fpga/xrt/lib/xleaf/clkfreq.c | 223 +++++ >> drivers/fpga/xrt/lib/xleaf/clock.c | 652 +++++++++++++ >> drivers/fpga/xrt/lib/xleaf/ddr_calibration.c | 210 +++++ >> drivers/fpga/xrt/lib/xleaf/devctl.c | 169 ++++ >> drivers/fpga/xrt/lib/xleaf/icap.c | 328 +++++++ >> drivers/fpga/xrt/lib/xleaf/ucs.c | 152 +++ >> drivers/fpga/xrt/lib/xleaf/vsec.c | 372 ++++++++ >> drivers/fpga/xrt/lib/xroot.c | 536 +++++++++++ >> drivers/fpga/xrt/metadata/Kconfig | 12 + >> drivers/fpga/xrt/metadata/Makefile | 16 + >> drivers/fpga/xrt/metadata/metadata.c | 578 ++++++++++++ >> drivers/fpga/xrt/mgmt/Kconfig | 15 + >> drivers/fpga/xrt/mgmt/Makefile | 19 + >> drivers/fpga/xrt/mgmt/root.c | 420 +++++++++ >> drivers/fpga/xrt/mgmt/xmgmt-main-region.c | 483 ++++++++++ >> drivers/fpga/xrt/mgmt/xmgmt-main.c | 662 +++++++++++++ >> drivers/fpga/xrt/mgmt/xmgmt.h | 33 + >> drivers/fpga/xrt/mgmt/xrt-mgr.c | 190 ++++ >> drivers/fpga/xrt/mgmt/xrt-mgr.h | 16 + >> include/uapi/linux/xrt/xclbin.h | 409 ++++++++ >> include/uapi/linux/xrt/xmgmt-ioctl.h | 46 + >> 53 files changed, 9988 insertions(+) >> create mode 100644 Documentation/fpga/xrt.rst >> create mode 100644 drivers/fpga/xrt/Kconfig >> create mode 100644 drivers/fpga/xrt/include/events.h >> create mode 100644 drivers/fpga/xrt/include/group.h >> create mode 100644 drivers/fpga/xrt/include/metadata.h >> create mode 100644 drivers/fpga/xrt/include/subdev_id.h >> create mode 100644 drivers/fpga/xrt/include/xclbin-helper.h >> create mode 100644 drivers/fpga/xrt/include/xdevice.h >> create mode 100644 drivers/fpga/xrt/include/xleaf.h >> create mode 100644 drivers/fpga/xrt/include/xleaf/axigate.h >> create mode 100644 drivers/fpga/xrt/include/xleaf/clkfreq.h >> create mode 100644 drivers/fpga/xrt/include/xleaf/clock.h >> create mode 100644 drivers/fpga/xrt/include/xleaf/ddr_calibration.h >> create mode 100644 drivers/fpga/xrt/include/xleaf/devctl.h >> create mode 100644 drivers/fpga/xrt/include/xleaf/icap.h >> create mode 100644 drivers/fpga/xrt/include/xmgmt-main.h >> create mode 100644 drivers/fpga/xrt/include/xroot.h >> create mode 100644 drivers/fpga/xrt/lib/Kconfig >> create mode 100644 drivers/fpga/xrt/lib/Makefile >> create mode 100644 drivers/fpga/xrt/lib/cdev.c >> create mode 100644 drivers/fpga/xrt/lib/group.c >> create mode 100644 drivers/fpga/xrt/lib/lib-drv.c >> create mode 100644 drivers/fpga/xrt/lib/lib-drv.h >> create mode 100644 drivers/fpga/xrt/lib/subdev.c >> create mode 100644 drivers/fpga/xrt/lib/subdev_pool.h >> create mode 100644 drivers/fpga/xrt/lib/xclbin.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/axigate.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/clkfreq.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/clock.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/ddr_calibration.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/devctl.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/icap.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/ucs.c >> create mode 100644 drivers/fpga/xrt/lib/xleaf/vsec.c >> create mode 100644 drivers/fpga/xrt/lib/xroot.c >> create mode 100644 drivers/fpga/xrt/metadata/Kconfig >> create mode 100644 drivers/fpga/xrt/metadata/Makefile >> create mode 100644 drivers/fpga/xrt/metadata/metadata.c >> create mode 100644 drivers/fpga/xrt/mgmt/Kconfig >> create mode 100644 drivers/fpga/xrt/mgmt/Makefile >> create mode 100644 drivers/fpga/xrt/mgmt/root.c >> create mode 100644 drivers/fpga/xrt/mgmt/xmgmt-main-region.c >> create mode 100644 drivers/fpga/xrt/mgmt/xmgmt-main.c >> create mode 100644 drivers/fpga/xrt/mgmt/xmgmt.h >> create mode 100644 drivers/fpga/xrt/mgmt/xrt-mgr.c >> create mode 100644 drivers/fpga/xrt/mgmt/xrt-mgr.h >> create mode 100644 include/uapi/linux/xrt/xclbin.h >> create mode 100644 include/uapi/linux/xrt/xmgmt-ioctl.h >> >
next prev parent reply other threads:[~2021-06-03 14:23 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-28 0:49 Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 01/20] Documentation: fpga: Add a document describing XRT Alveo drivers Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 02/20] fpga: xrt: driver metadata helper functions Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 03/20] fpga: xrt: xclbin file " Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 04/20] fpga: xrt: xrt-lib driver manager Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 05/20] fpga: xrt: group driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 06/20] fpga: xrt: char dev node helper functions Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 07/20] fpga: xrt: root driver infrastructure Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 08/20] fpga: xrt: " Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 09/20] fpga: xrt: management physical function driver (root) Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 10/20] fpga: xrt: main driver for management function device Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 11/20] fpga: xrt: fpga-mgr and region implementation for xclbin download Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 12/20] fpga: xrt: VSEC driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 13/20] fpga: xrt: User Clock Subsystem driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 14/20] fpga: xrt: ICAP driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 15/20] fpga: xrt: devctl xrt driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 16/20] fpga: xrt: clock driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 17/20] fpga: xrt: clock frequency counter driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 18/20] fpga: xrt: DDR calibration driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 19/20] fpga: xrt: partition isolation driver Lizhi Hou 2021-05-28 0:49 ` [PATCH V7 XRT Alveo 20/20] fpga: xrt: Kconfig and Makefile updates for XRT drivers Lizhi Hou 2021-06-02 22:34 ` [PATCH V7 XRT Alveo 00/20] XRT Alveo driver overview Lizhi Hou 2021-06-03 14:23 ` Tom Rix [this message] 2021-06-10 23:36 ` RFC : " Lizhi Hou 2021-06-11 12:46 ` Tom Rix 2021-07-01 20:32 ` Tom Rix 2021-07-16 18:27 ` Tom Rix 2021-07-16 19:14 ` Lizhi Hou 2021-07-16 20:36 ` Tom Rix 2021-07-19 21:22 ` Lizhi Hou
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=68e85a4f-4a10-1ff9-0443-aa565878c855@redhat.com \ --to=trix@redhat.com \ --cc=devicetree@vger.kernel.org \ --cc=hao.wu@intel.com \ --cc=linux-fpga@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=lizhi.hou@xilinx.com \ --cc=maxz@xilinx.com \ --cc=mdf@kernel.org \ --cc=michal.simek@xilinx.com \ --cc=robh@kernel.org \ --cc=sonal.santan@xilinx.com \ --cc=stefanos@xilinx.com \ --cc=yliu@xilinx.com \ --subject='Re: RFC : [PATCH V7 XRT Alveo 00/20] XRT Alveo driver overview' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).