From: Wu Hao <hao.wu@intel.com>
To: Alan Tull <atull@kernel.org>
Cc: Moritz Fischer <mdf@kernel.org>,
linux-fpga@vger.kernel.org,
linux-kernel <linux-kernel@vger.kernel.org>,
linux-api@vger.kernel.org, "Kang, Luwei" <luwei.kang@intel.com>,
"Zhang, Yi Z" <yi.z.zhang@intel.com>,
Tim Whisonant <tim.whisonant@intel.com>,
Enno Luebbers <enno.luebbers@intel.com>,
Shiva Rao <shiva.rao@intel.com>,
Christopher Rauer <christopher.rauer@intel.com>,
Xiao Guangrong <guangrong.xiao@linux.intel.com>
Subject: Re: [PATCH v4 09/24] fpga: dfl-pci: add enumeration for feature devices
Date: Wed, 14 Mar 2018 13:21:21 +0800 [thread overview]
Message-ID: <20180314052121.GB32276@hao-dev> (raw)
In-Reply-To: <CANk1AXSfXBXj_NH4Qcj6EDXEzAD_+T1sVudJdMbbgKK7hk13Mg@mail.gmail.com>
On Tue, Mar 13, 2018 at 01:30:24PM -0500, Alan Tull wrote:
> On Tue, Feb 13, 2018 at 3:24 AM, Wu Hao <hao.wu@intel.com> wrote:
>
> Hi Hao,
>
> Thanks again for splitting the pci part of the code from enumeration
> and everything else.
>
> One thing that may need to be fixed below, so with that fixed, adding my ack.
Hi Alan
Thanks a lot for your review and acked-by on these patches,
please see my replies below. : )
>
> > The Device Feature List (DFL) is implemented in MMIO, and features
> > are linked via the DFLs. This patch enables pcie driver to prepare
> > enumeration information (e.g locations of all device feature lists
> > in MMIO) and use common APIs provided by the Device Feature List
> > framework to enumerate each feature device linked.
> >
> > Signed-off-by: Tim Whisonant <tim.whisonant@intel.com>
> > Signed-off-by: Enno Luebbers <enno.luebbers@intel.com>
> > Signed-off-by: Shiva Rao <shiva.rao@intel.com>
> > Signed-off-by: Christopher Rauer <christopher.rauer@intel.com>
> > Signed-off-by: Zhang Yi <yi.z.zhang@intel.com>
> > Signed-off-by: Xiao Guangrong <guangrong.xiao@linux.intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> Acked-by: Alan Tull <atull@kernel.org>
>
> > ---
> > v3: split from another patch
> > use common functions from DFL framework for enumeration.
> > v4: rebase
> > ---
> > drivers/fpga/dfl-pci.c | 199 ++++++++++++++++++++++++++++++++++++++++++++++++-
> > 1 file changed, 197 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c
> > index d91ea42..8ce8a94 100644
> > --- a/drivers/fpga/dfl-pci.c
> > +++ b/drivers/fpga/dfl-pci.c
> > @@ -22,9 +22,52 @@
> > #include <linux/errno.h>
> > #include <linux/aer.h>
> >
> > +#include "dfl.h"
> > +
> > #define DRV_VERSION "0.8"
> > #define DRV_NAME "dfl-pci"
> >
> > +struct cci_drvdata {
> > + struct fpga_cdev *cdev; /* container device */
> > + struct list_head regions; /* list of pci bar mapping region */
> > +};
> > +
> > +/* pci bar mapping info */
> > +struct cci_region {
> > + int bar;
> > + void __iomem *ioaddr; /* pointer to mapped bar region */
> > + struct list_head node;
> > +};
> > +
> > +static void __iomem *cci_pci_ioremap_bar(struct pci_dev *pcidev, int bar)
> > +{
> > + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> > + struct cci_region *region;
> > +
> > + list_for_each_entry(region, &drvdata->regions, node)
> > + if (region->bar == bar) {
> > + dev_dbg(&pcidev->dev, "BAR %d region exists\n", bar);
> > + return region->ioaddr;
> > + }
> > +
> > + region = devm_kzalloc(&pcidev->dev, sizeof(*region), GFP_KERNEL);
> > + if (!region)
> > + return NULL;
> > +
> > + region->bar = bar;
> > + region->ioaddr = pci_ioremap_bar(pcidev, bar);
> > + if (!region->ioaddr) {
> > + dev_err(&pcidev->dev, "can't ioremap memory from BAR %d.\n",
> > + bar);
> > + devm_kfree(&pcidev->dev, region);
> > + return NULL;
> > + }
> > +
> > + list_add(®ion->node, &drvdata->regions);
> > +
> > + return region->ioaddr;
> > +}
> > +
> > /* PCI Device ID */
> > #define PCIE_DEVICE_ID_PF_INT_5_X 0xBCBD
> > #define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0
> > @@ -45,6 +88,143 @@ static struct pci_device_id cci_pcie_id_tbl[] = {
> > };
> > MODULE_DEVICE_TABLE(pci, cci_pcie_id_tbl);
> >
> > +static int cci_init_drvdata(struct pci_dev *pcidev)
> > +{
> > + struct cci_drvdata *drvdata;
> > +
> > + drvdata = devm_kzalloc(&pcidev->dev, sizeof(*drvdata), GFP_KERNEL);
> > + if (!drvdata)
> > + return -ENOMEM;
> > +
> > + INIT_LIST_HEAD(&drvdata->regions);
> > +
> > + pci_set_drvdata(pcidev, drvdata);
> > +
> > + return 0;
> > +}
> > +
> > +static void cci_pci_release_regions(struct pci_dev *pcidev)
> > +{
> > + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> > + struct cci_region *tmp, *region;
> > +
> > + list_for_each_entry_safe(region, tmp, &drvdata->regions, node) {
> > + list_del(®ion->node);
> > + if (region->ioaddr)
> > + pci_iounmap(pcidev, region->ioaddr);
> > + devm_kfree(&pcidev->dev, region);
> > + }
> > +}
> > +
> > +static void cci_remove_drvdata(struct pci_dev *pcidev)
> > +{
> > + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> > +
> > + cci_pci_release_regions(pcidev);
>
> Would it make sense to call fpga_enum_info_free here? I understand
> fpga_enum_info_alloc uses devm, but it does a get_device which needs
> to be put.
>
> > + pci_set_drvdata(pcidev, NULL);
> > + devm_kfree(&pcidev->dev, drvdata);
> > +}
> > +
> > +static void cci_remove_feature_devs(struct pci_dev *pcidev)
> > +{
> > + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> > +
> > + /* remove all children feature devices */
> > + fpga_remove_feature_devs(drvdata->cdev);
> > +}
> > +
> > +/* enumerate feature devices under pci device */
> > +static int cci_enumerate_feature_devs(struct pci_dev *pcidev)
> > +{
> > + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> > + struct fpga_cdev *cdev;
> > + struct fpga_enum_info *info;
> > + resource_size_t start, len;
> > + void __iomem *base;
> > + int port_num, bar, i, ret = 0;
> > + u32 offset;
> > + u64 v;
> > +
> > + /* allocate enumeration info via pci_dev */
> > + info = fpga_enum_info_alloc(&pcidev->dev);
> > + if (!info)
> > + return -ENOMEM;
> > +
> > + /* start to find Device Feature List from Bar 0 */
> > + base = cci_pci_ioremap_bar(pcidev, 0);
> > + if (!base) {
> > + ret = -ENOMEM;
> > + goto enum_info_free_exit;
> > + }
> > +
> > + /*
> > + * PF device has FME and Ports/AFUs, and VF device only has 1 Port/AFU.
> > + * check them and add related "Device Feature List" info for the next
> > + * step enumeration.
> > + */
> > + if (feature_is_fme(base)) {
> > + start = pci_resource_start(pcidev, 0);
> > + len = pci_resource_len(pcidev, 0);
> > +
> > + fpga_enum_info_add_dfl(info, start, len, base);
> > +
> > + /*
> > + * find more Device Feature Lists (e.g Ports) per information
> > + * indicated by FME module.
> > + */
> > + v = readq(base + FME_HDR_CAP);
> > + port_num = FIELD_GET(FME_CAP_NUM_PORTS, v);
> > +
> > + WARN_ON(port_num > MAX_FPGA_PORT_NUM);
> > +
> > + for (i = 0; i < port_num; i++) {
> > + v = readq(base + FME_HDR_PORT_OFST(i));
> > +
> > + /* skip ports which are not implemented. */
> > + if (!(v & FME_PORT_OFST_IMP))
> > + continue;
> > +
> > + /*
> > + * add Port's Device Feature List information for next
> > + * step enumeration.
> > + */
> > + bar = FIELD_GET(FME_PORT_OFST_BAR_ID, v);
> > + offset = FIELD_GET(FME_PORT_OFST_DFH_OFST, v);
> > + base = cci_pci_ioremap_bar(pcidev, bar);
> > + if (!base)
> > + continue;
> > +
> > + start = pci_resource_start(pcidev, bar) + offset;
> > + len = pci_resource_len(pcidev, bar) - offset;
> > +
> > + fpga_enum_info_add_dfl(info, start, len, base + offset);
> > + }
> > + } else if (feature_is_port(base)) {
> > + start = pci_resource_start(pcidev, 0);
> > + len = pci_resource_len(pcidev, 0);
> > +
> > + fpga_enum_info_add_dfl(info, start, len, base);
> > + } else {
> > + ret = -ENODEV;
> > + goto enum_info_free_exit;
> > + }
> > +
> > + /* start enumeration with prepared enumeration information */
> > + cdev = fpga_enumerate_feature_devs(info);
> > + if (IS_ERR(cdev)) {
> > + dev_err(&pcidev->dev, "Enumeration failure\n");
> > + ret = PTR_ERR(cdev);
> > + goto enum_info_free_exit;
> > + }
> > +
> > + drvdata->cdev = cdev;
> > +
> > +enum_info_free_exit:
> > + fpga_enum_info_free(info);
>
> This is the only place I saw fpga_enum_info_free being called.
It doesn't need to keep the enumeration inforamtion data structure once
the enumeration done, so in the driver, it always did fpga_enum_info_free
once fpga_enumerate_feature_devs(info) returned in this function. so
no need to consider it in other places per my understanding. : )
Thanks
Hao
>
> Thanks,
> Alan
>
> > +
> > + return ret;
> > +}
> > +
> > static
> > int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid)
> > {
> > @@ -82,9 +262,22 @@ int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid)
> > goto release_region_exit;
> > }
> >
> > - /* TODO: create and add the platform device per feature list */
> > - return 0;
> > + ret = cci_init_drvdata(pcidev);
> > + if (ret) {
> > + dev_err(&pcidev->dev, "Fail to init drvdata %d.\n", ret);
> > + goto release_region_exit;
> > + }
> > +
> > + ret = cci_enumerate_feature_devs(pcidev);
> > + if (ret) {
> > + dev_err(&pcidev->dev, "enumeration failure %d.\n", ret);
> > + goto remove_drvdata_exit;
> > + }
> > +
> > + return ret;
> >
> > +remove_drvdata_exit:
> > + cci_remove_drvdata(pcidev);
> > release_region_exit:
> > pci_release_regions(pcidev);
> > disable_error_report_exit:
> > @@ -95,6 +288,8 @@ int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid)
> >
> > static void cci_pci_remove(struct pci_dev *pcidev)
> > {
> > + cci_remove_feature_devs(pcidev);
> > + cci_remove_drvdata(pcidev);
> > pci_release_regions(pcidev);
> > pci_disable_pcie_error_reporting(pcidev);
> > pci_disable_device(pcidev);
> > --
> > 2.7.4
> >
next prev parent reply other threads:[~2018-03-14 5:31 UTC|newest]
Thread overview: 93+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-13 9:24 [PATCH v4 00/24] FPGA Device Feature List (DFL) Device Drivers Wu Hao
2018-02-13 9:24 ` [PATCH v4 01/24] docs: fpga: add a document for FPGA Device Feature List (DFL) Framework Overview Wu Hao
2018-02-26 22:48 ` Alan Tull
2018-02-27 2:12 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 02/24] fpga: mgr: add region_id to fpga_image_info Wu Hao
2018-02-13 9:24 ` [PATCH v4 03/24] fpga: mgr: add status for fpga-manager Wu Hao
2018-02-14 15:55 ` Alan Tull
2018-02-15 9:42 ` Wu, Hao
2018-02-14 20:55 ` Moritz Fischer
2018-02-13 9:24 ` [PATCH v4 04/24] fpga: add device feature list support Wu Hao
2018-03-21 23:54 ` Alan Tull
2018-03-22 4:40 ` Wu Hao
2018-03-22 21:31 ` Alan Tull
2018-03-23 4:33 ` Wu Hao
2018-03-26 17:21 ` Alan Tull
2018-03-27 2:35 ` Wu Hao
2018-03-29 21:57 ` Alan Tull
2018-04-02 4:22 ` Wu Hao
2018-04-02 19:06 ` Alan Tull
2018-04-03 1:36 ` Wu Hao
2018-04-04 20:06 ` Alan Tull
2018-04-06 11:01 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 05/24] fpga: dfl: add chardev support for feature devices Wu Hao
2018-02-13 9:24 ` [PATCH v4 06/24] fpga: dfl: adds fpga_cdev_find_port Wu Hao
2018-02-14 16:24 ` Alan Tull
2018-02-15 9:46 ` Wu, Hao
2018-02-14 20:55 ` Moritz Fischer
2018-02-13 9:24 ` [PATCH v4 07/24] fpga: dfl: add feature device infrastructure Wu Hao
2018-02-14 21:03 ` Moritz Fischer
2018-02-14 21:13 ` Alan Tull
2018-02-15 10:05 ` Wu, Hao
2018-02-15 19:49 ` Moritz Fischer
2018-02-18 2:15 ` Wu, Hao
2018-02-13 9:24 ` [PATCH v4 08/24] fpga: add FPGA DFL PCIe device driver Wu Hao
2018-03-13 16:05 ` Alan Tull
2018-03-15 18:49 ` Moritz Fischer
2018-03-16 4:29 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 09/24] fpga: dfl-pci: add enumeration for feature devices Wu Hao
2018-03-13 18:30 ` Alan Tull
2018-03-14 5:21 ` Wu Hao [this message]
2018-03-14 14:48 ` Alan Tull
2018-02-13 9:24 ` [PATCH v4 10/24] fpga: dfl: add FPGA Management Engine driver basic framework Wu Hao
2018-04-05 18:35 ` Alan Tull
2018-04-06 11:04 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 11/24] fpga: dfl: fme: add header sub feature support Wu Hao
2018-02-14 16:36 ` Alan Tull
2018-02-13 9:24 ` [PATCH v4 12/24] fpga: dfl: fme: add FPGA_GET_API_VERSION/CHECK_EXTENSION ioctls support Wu Hao
2018-03-19 18:29 ` Alan Tull
2018-03-20 6:46 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 13/24] fpga: region: add compat_id support Wu Hao
2018-02-28 22:55 ` Alan Tull
2018-03-01 6:17 ` Wu Hao
2018-03-05 19:42 ` Alan Tull
2018-03-06 0:56 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 14/24] fpga: dfl: fme: add partial reconfiguration sub feature support Wu Hao
2018-03-05 22:46 ` Alan Tull
2018-03-06 2:08 ` Wu Hao
2018-03-06 18:29 ` Alan Tull
2018-03-07 4:39 ` Wu Hao
2018-03-11 20:09 ` matthew.gerlach
2018-03-12 4:29 ` Wu Hao
2018-03-12 18:53 ` Alan Tull
2018-03-12 21:36 ` matthew.gerlach
2018-03-13 1:07 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 15/24] fpga: dfl-fme-pr: add compat_id support for dfl-fme-region platform device Wu Hao
2018-02-28 23:06 ` Alan Tull
2018-03-01 5:49 ` Wu Hao
2018-03-01 15:59 ` Alan Tull
2018-03-01 15:55 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 16/24] fpga: dfl: add fpga manager platform driver for FME Wu Hao
2018-03-20 20:32 ` Alan Tull
2018-03-21 2:50 ` Wu Hao
2018-03-21 16:55 ` Moritz Fischer
2018-03-22 6:07 ` Wu Hao
2018-04-05 18:45 ` Alan Tull
2018-04-06 11:11 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 17/24] fpga: dfl: add fpga bridge " Wu Hao
2018-02-13 9:24 ` [PATCH v4 18/24] fpga: dfl: add fpga region " Wu Hao
2018-02-13 9:24 ` [PATCH v4 19/24] fpga: dfl-fme-region: add compat_id support Wu Hao
2018-02-13 9:24 ` [PATCH v4 20/24] fpga: dfl: add FPGA Accelerated Function Unit driver basic framework Wu Hao
2018-03-19 18:40 ` Alan Tull
2018-04-05 18:26 ` Alan Tull
2018-04-06 11:05 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 21/24] fpga: dfl: afu: add header sub feature support Wu Hao
2018-02-13 9:24 ` [PATCH v4 22/24] fpga: dfl: afu: add FPGA_GET_API_VERSION/CHECK_EXTENSION ioctls support Wu Hao
2018-02-13 9:24 ` [PATCH v4 23/24] fpga: dfl: afu: add user afu sub feature support Wu Hao
2018-03-19 20:10 ` Alan Tull
2018-03-20 7:10 ` Wu Hao
2018-03-20 18:17 ` Alan Tull
2018-03-21 3:00 ` Wu Hao
2018-03-21 23:50 ` Alan Tull
2018-03-22 4:41 ` Wu Hao
2018-02-13 9:24 ` [PATCH v4 24/24] fpga: dfl: afu: add FPGA_PORT_DMA_MAP/UNMAP ioctls support Wu Hao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180314052121.GB32276@hao-dev \
--to=hao.wu@intel.com \
--cc=atull@kernel.org \
--cc=christopher.rauer@intel.com \
--cc=enno.luebbers@intel.com \
--cc=guangrong.xiao@linux.intel.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-fpga@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luwei.kang@intel.com \
--cc=mdf@kernel.org \
--cc=shiva.rao@intel.com \
--cc=tim.whisonant@intel.com \
--cc=yi.z.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).