RE: [PATCH 2/5] misc: zynq: Add afi config driver

[Nava kishore Manne <navam@xilinx.com>]

Hi Greg,

	Please find my response inline.

> -----Original Message-----
> From: Greg KH <gregkh@linuxfoundation.org>
> Sent: Tuesday, April 20, 2021 8:57 PM
> To: Nava kishore Manne <navam@xilinx.com>
> Cc: robh+dt@kernel.org; Michal Simek <michals@xilinx.com>; Derek Kiernan
> <dkiernan@xilinx.com>; Dragan Cvetic <draganc@xilinx.com>;
> arnd@arndb.de; Rajan Vaja <RAJANV@xilinx.com>; Jolly Shah
> <JOLLYS@xilinx.com>; Tejas Patel <tejasp@xlnx.xilinx.com>; Amit Sunil
> Dhamne <amitsuni@xilinx.com>; devicetree@vger.kernel.org; linux-arm-
> kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> chinnikishore369@gmail.com; git <git@xilinx.com>
> Subject: Re: [PATCH 2/5] misc: zynq: Add afi config driver
> 
> On Tue, Apr 20, 2021 at 01:36:51PM +0000, Nava kishore Manne wrote:
> > Hi Greg,
> >
> > 	Please find my response inline.
> >
> > > -----Original Message-----
> > > From: Greg KH <gregkh@linuxfoundation.org>
> > > Sent: Tuesday, April 20, 2021 2:17 PM
> > > To: Nava kishore Manne <navam@xilinx.com>
> > > Cc: robh+dt@kernel.org; Michal Simek <michals@xilinx.com>; Derek
> > > Kiernan <dkiernan@xilinx.com>; Dragan Cvetic <draganc@xilinx.com>;
> > > arnd@arndb.de; Rajan Vaja <RAJANV@xilinx.com>; Jolly Shah
> > > <JOLLYS@xilinx.com>; Tejas Patel <tejasp@xlnx.xilinx.com>; Amit
> > > Sunil Dhamne <amitsuni@xilinx.com>; devicetree@vger.kernel.org;
> > > linux-arm- kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> > > chinnikishore369@gmail.com; git <git@xilinx.com>
> > > Subject: Re: [PATCH 2/5] misc: zynq: Add afi config driver
> > >
> > > On Tue, Apr 20, 2021 at 01:41:50PM +0530, Nava kishore Manne wrote:
> > > > This patch adds zynq afi config driver. This is useful for the
> > > > configuration of the PS-PL interface on zynq platform.
> > >
> > > What is "PS-PL"?  Can you describe it better please?
> > >
> > PS-PL interface is nothing but the interface between processing system(PS)
> that contains arm cores and Programmable Logic(PL) i.e FPGA.
> > Will update the description in v2.
> >
> > > >
> > > > Signed-off-by: Nava kishore Manne <nava.manne@xilinx.com>
> > > > ---
> > > >  drivers/misc/Kconfig    | 11 ++++++
> > > >  drivers/misc/Makefile   |  1 +
> > > >  drivers/misc/zynq-afi.c | 81
> > > > +++++++++++++++++++++++++++++++++++++++++
> > > >  3 files changed, 93 insertions(+)  create mode 100644
> > > > drivers/misc/zynq-afi.c
> > > >
> > > > diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index
> > > > f532c59bb59b..877b43b3377d 100644
> > > > --- a/drivers/misc/Kconfig
> > > > +++ b/drivers/misc/Kconfig
> > > > @@ -445,6 +445,17 @@ config HISI_HIKEY_USB
> > > >  	  switching between the dual-role USB-C port and the USB-A host
> > > ports
> > > >  	  using only one USB controller.
> > > >
> > > > +config ZYNQ_AFI
> > > > +	tristate "Xilinx ZYNQ AFI support"
> > > > +	help
> > > > +	  Zynq AFI driver support for writing to the AFI registers
> > > > +	  for configuring PS_PL Bus-width. Xilinx Zynq SoC connect
> > > > +	  the PS to the programmable logic (PL) through the AXI port.
> > > > +	  This AXI port helps to establish the data path between the
> > > > +	  PS and PL.In-order to establish the proper communication path
> > > > +	  between PS and PL, the AXI port data path should be configured
> > > > +	  with the proper Bus-width values
> > > > +
> > > >  source "drivers/misc/c2port/Kconfig"
> > > >  source "drivers/misc/eeprom/Kconfig"
> > > >  source "drivers/misc/cb710/Kconfig"
> > > > diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index
> > > > 99b6f15a3c70..e9b03843100f 100644
> > > > --- a/drivers/misc/Makefile
> > > > +++ b/drivers/misc/Makefile
> > > > @@ -56,3 +56,4 @@ obj-$(CONFIG_HABANA_AI)		+=
> > > habanalabs/
> > > >  obj-$(CONFIG_UACCE)		+= uacce/
> > > >  obj-$(CONFIG_XILINX_SDFEC)	+= xilinx_sdfec.o
> > > >  obj-$(CONFIG_HISI_HIKEY_USB)	+= hisi_hikey_usb.o
> > > > +obj-$(CONFIG_ZYNQ_AFI)		+= zynq-afi.o
> > > > diff --git a/drivers/misc/zynq-afi.c b/drivers/misc/zynq-afi.c new
> > > > file mode 100644 index 000000000000..04317d1bdb98
> > > > --- /dev/null
> > > > +++ b/drivers/misc/zynq-afi.c
> > > > @@ -0,0 +1,81 @@
> > > > +// SPDX-License-Identifier: GPL-2.0
> > > > +/*
> > > > + * Xilinx ZYNQ AFI driver.
> > > > + * Copyright (c) 2018-2021 Xilinx Inc.
> > > > + */
> > > > +
> > > > +#include <linux/err.h>
> > > > +#include <linux/io.h>
> > > > +#include <linux/module.h>
> > > > +#include <linux/of.h>
> > > > +#include <linux/platform_device.h>
> > > > +
> > > > +/* Registers and special values for doing register-based operations */
> > > > +#define AFI_RDCHAN_CTRL_OFFSET	0x00
> > > > +#define AFI_WRCHAN_CTRL_OFFSET	0x14
> > > > +
> > > > +#define AFI_BUSWIDTH_MASK	0x01
> > > > +
> > > > +/**
> > > > + * struct afi_fpga - AFI register description
> > > > + * @membase:	pointer to register struct
> > > > + * @afi_width:	AFI bus width to be written
> > > > + */
> > > > +struct zynq_afi_fpga {
> > > > +	void __iomem	*membase;
> > > > +	u32		afi_width;
> > > > +};
> > > > +
> > > > +static int zynq_afi_fpga_probe(struct platform_device *pdev) {
> > > > +	struct zynq_afi_fpga *afi_fpga;
> > > > +	struct resource *res;
> > > > +	u32 reg_val;
> > > > +	u32 val;
> > > > +
> > > > +	afi_fpga = devm_kzalloc(&pdev->dev, sizeof(*afi_fpga),
> > > GFP_KERNEL);
> > > > +	if (!afi_fpga)
> > > > +		return -ENOMEM;
> > > > +
> > > > +	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> > > > +	afi_fpga->membase = devm_ioremap_resource(&pdev->dev, res);
> > > > +	if (IS_ERR(afi_fpga->membase))
> > > > +		return PTR_ERR(afi_fpga->membase);
> > > > +
> > > > +	val = device_property_read_u32(&pdev->dev, "xlnx,afi-width",
> > > > +				       &afi_fpga->afi_width);
> > > > +	if (val) {
> > > > +		dev_err(&pdev->dev, "failed to get the afi bus width\n");
> > > > +		return -EINVAL;
> > > > +	}
> > > > +
> > > > +	reg_val = readl(afi_fpga->membase + AFI_RDCHAN_CTRL_OFFSET);
> > > > +	reg_val &= ~AFI_BUSWIDTH_MASK;
> > > > +	writel(reg_val | afi_fpga->afi_width,
> > > > +	       afi_fpga->membase + AFI_RDCHAN_CTRL_OFFSET);
> > > > +	reg_val = readl(afi_fpga->membase + AFI_WRCHAN_CTRL_OFFSET);
> > > > +	reg_val &= ~AFI_BUSWIDTH_MASK;
> > > > +	writel(reg_val | afi_fpga->afi_width,
> > > > +	       afi_fpga->membase + AFI_WRCHAN_CTRL_OFFSET);
> > > > +
> > > > +	return 0;
> > > > +}
> > >
> > > I do not understand, why is this driver needed at all?  Why can't
> > > you do the above from userspace?
> > >
> > > All this does is write some values to the hardware at probe time,
> > > who needs this?
> >
> > This driver will be used by the overlay framework for configuring the
> interface after programming the FPGA and before probing the drivers that
> are present in the PL.
> 
> What is a "overlay framework"?  And why does the kernel have to do this?
> Why can't you write these hardware values from userspace?
> 
> confused,
> 

The Zynq based processing system (PS) that contains ARM cores and Xilinx programmable logic (PL/FPGA) in a single device. The PS and PL can be tightly or loosely coupled using multiple interfaces and other signals.
This enables the designer to effectively integrate user-created hardware accelerators and other functions in the PL logic that are accessible to the processors and can also access memory resources in the PS.
https://www.xilinx.com/support/documentation/user_guides/ug585-Zynq-7000-TRM.pdf (Page No: 41)

To Program/Re-Program the PL at runtime in Linux we have a an FPGA Manger Framework and this frame work uses DT-Overlays to programming the FPGA and probing the relevant PL drivers.
For more info please refer this link: https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/fpga/fpga-region.txt   

Every Zynq/ZynqMP PL(FPGA) design has its own PS-PL configuration. So after programming the FPGA and before probing the relevant PL drivers these PS-PL configurations should be set.

Example DT-Overlay file to program the PL(FPGA) from Linux:
/dts-v1/;
/plugin/;
/ {
        fragment@0 {  /* fragment 0 contains Bitstream info */
                target = <&fpga_full>;
                overlay0: __overlay__ {
                        #address-cells = <2>;
                        #size-cells = <2>;
                        firmware-name = "Base_Zynq_MPSoC_wrapper.bit.bin";
                        resets = <&zynqmp_reset 116>;
                };
        };
        fragment@1 { /* fragment 1 contains PS-PL configurations */
                target = <&amba>;
                overlay1: __overlay__ {
                        afi0: afi@f8008000 {
                                #address-cells = <0x1>;
                                #size-cells = <0x0>;
                                compatible = "xlnx,zynq-afi-fpga";
                                reg = <0xf8008000 0x24>;
                                xlnx,afi-width = <0x1>;
                        };
                 };
        };
       fragment@2 { /* Fragment 2 contains the relevant drivers  for the IP's present in the FPGA design*/
                target = <&amba>;
                overlay2: __overlay__ {
                        axi_gpio_0: gpio@a0000000 {
                                #gpio-cells = <2>;
                                clock-names = "s_axi_aclk";
                                clocks = <&zynqmp_clk 71>;
                                compatible = "xlnx,xps-gpio-1.00.a";
                                gpio-controller ;
                                reg = <0x0 0xa0000000 0x0 0x1000>;
                        };
                };
         } ;               
};

In-order to support the PL(FPGA) programming and to configure the interface between PS and PL using FPGA Manager. This Driver is  needed in the kernel space.

@Moritz Fischer: Can you please let us know your thoughts on this.

Regards,
Navakishore.