From: Greg KH <gregkh@linuxfoundation.org>
To: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Tianfei Zhang <tianfei.zhang@intel.com>,
bhelgaas@google.com, linux-pci@vger.kernel.org,
linux-fpga@vger.kernel.org, lukas@wunner.de, kabel@kernel.org,
mani@kernel.org, pali@kernel.org, mdf@kernel.org,
hao.wu@intel.com, yilun.xu@intel.com, trix@redhat.com,
jgg@ziepe.ca, ira.weiny@intel.com,
andriy.shevchenko@linux.intel.com, dan.j.williams@intel.com,
keescook@chromium.org, russell.h.weight@intel.com,
corbet@lwn.net, linux-doc@vger.kernel.org,
ilpo.jarvinen@linux.intel.com, lee@kernel.org,
matthew.gerlach@linux.intel.com
Subject: Re: [PATCH v1 00/12] add FPGA hotplug manager driver
Date: Thu, 19 Jan 2023 16:33:05 +0100 [thread overview]
Message-ID: <Y8lisT7xmkd1/g+J@kroah.com> (raw)
In-Reply-To: <CAJZ5v0jW28k5+CN_F4dLo=OzVVJEL+U=gW4bzeSPjU=j53BJKg@mail.gmail.com>
On Thu, Jan 19, 2023 at 02:43:21PM +0100, Rafael J. Wysocki wrote:
> On Thu, Jan 19, 2023 at 2:34 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Wed, Jan 18, 2023 at 08:35:50PM -0500, Tianfei Zhang wrote:
> > > This patchset introduces the FPGA hotplug manager (fpgahp) driver which
> > > has been verified on the Intel N3000 card.
> > >
> > > When a PCIe-based FPGA card is reprogrammed, it temporarily disappears
> > > from the PCIe bus. This needs to be managed to avoid PCIe errors and to
> > > reprobe the device after reprogramming.
> > >
> > > To change the FPGA image, the kernel burns a new image into the flash on
> > > the card, and then triggers the card BMC to load the new image into FPGA.
> > > A new FPGA hotplug manager driver is introduced that leverages the PCIe
> > > hotplug framework to trigger and manage the update of the FPGA image,
> > > including the disappearance and reappearance of the card on the PCIe bus.
> > > The fpgahp driver uses APIs from the pciehp driver. Two new operation
> > > callbacks are defined in hotplug_slot_ops:
> > >
> > > - available_images: Optional: available FPGA images
> > > - image_load: Optional: trigger the FPGA to load a new image
> > >
> > >
> > > The process of reprogramming an FPGA card begins by removing all devices
> > > associated with the card that are not required for the reprogramming of
> > > the card. This includes PCIe devices (PFs and VFs) associated with the
> > > card as well as any other types of devices (platform, etc.) defined within
> > > the FPGA. The remaining devices are referred to here as "reserved" devices.
> > > After triggering the update of the FPGA card, the reserved devices are also
> > > removed.
> > >
> > > The complete process for reprogramming the FPGA are:
> > > 1. remove all PFs and VFs except for PF0 (reserved).
> > > 2. remove all non-reserved devices of PF0.
> > > 3. trigger FPGA card to do the image update.
> > > 4. disable the link of the hotplug bridge.
> > > 5. remove all reserved devices under hotplug bridge.
> > > 6. wait for image reload done via BMC, e.g. 10s.
> > > 7. re-enable the link of hotplug bridge
> > > 8. enumerate PCI devices below the hotplug bridge
> > >
> > > usage example:
> > > [root@localhost]# cd /sys/bus/pci/slot/X-X/
> > >
> > > Get the available images.
> > > [root@localhost 2-1]# cat available_images
> > > bmc_factory bmc_user retimer_fw
> > >
> > > Load the request images for FPGA Card, for example load the BMC user image:
> > > [root@localhost 2-1]# echo bmc_user > image_load
> >
> > Why is all of this tied into the pci hotplug code? Shouldn't it be
> > specific to this one driver instead? pci hotplug is for removing/adding
> > PCI devices to the system, not messing with FPGA images.
> >
> > This feels like an abuse of the pci hotplug bus to me as this is NOT
> > really a PCI hotplug bus at all, right?
> >
> > Or is it? If so, then the slots should show up under the PCI device
> > itself, not in /sys/bus/pci/slot/. That location is there for old old
> > stuff, we probably should move it one of these days as there's lots of
> > special-cases in the driver core just because of that :(
>
> I'm not sure if I can agree with this statement.
>
> The slot here is what is registered via pci_hp_register(), isn't it?
Yes, but is it really a "slot" like a normal PCI slot?
> There are multiple users of this in the tree, including ACPI-based PCI
> hotplug, which is not really that old.
It's really old, I think I worked on that in the 2.4/2.5 days? Anyway,
it's been around a long time.
> Are you saying that this should not be used?
I'm saying that PCI is the only subsystem/bus that has something like
this and we have a number of functions exported in the driver core only
for the pci hotplug slot list. Which kind of implies that maybe it
should be moved to something else? I don't have any specific ideas what
it should be, just that it feels really odd as-is still.
thanks,
greg k-h
next prev parent reply other threads:[~2023-01-19 15:36 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-19 1:35 [PATCH v1 00/12] add FPGA hotplug manager driver Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 01/12] PCI: hotplug: add new callbacks on hotplug_slot_ops Tianfei Zhang
2023-01-19 13:31 ` Greg KH
2023-01-19 1:35 ` [PATCH v1 02/12] PCI: hotplug: expose APIs from pciehp driver Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 03/12] PCI: hotplug: add and expose link disable API Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 04/12] PCI: hotplug: add FPGA PCI hotplug manager driver Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 05/12] fpga: dfl: register dfl-pci device into fpgahph driver Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 06/12] driver core: expose device_is_ancestor() API Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 07/12] PCI: hotplug: add register/unregister function for BMC device Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 08/12] fpga: m10bmc-sec: register BMC device into fpgahp driver Tianfei Zhang
2023-01-19 1:35 ` [PATCH v1 09/12] fpga: dfl: remove non-reserved devices Tianfei Zhang
2023-01-19 1:36 ` [PATCH v1 10/12] PCI: hotplug: implement the hotplug_slot_ops callback for fpgahp Tianfei Zhang
2023-01-19 13:28 ` Greg KH
2023-01-20 22:38 ` Russ Weight
2023-01-21 7:35 ` Greg KH
2023-01-19 1:36 ` [PATCH v1 11/12] fpga: m10bmc-sec: add m10bmc_sec_retimer_load callback Tianfei Zhang
2023-01-19 14:22 ` Lee Jones
2023-01-19 1:36 ` [PATCH v1 12/12] Documentation: fpga: add description of fpgahp driver Tianfei Zhang
2023-01-19 9:38 ` Bagas Sanjaya
2023-01-19 8:06 ` [PATCH v1 00/12] add FPGA hotplug manager driver Pali Rohár
2023-01-19 8:17 ` Zhang, Tianfei
2023-01-19 11:27 ` andriy.shevchenko
2023-01-19 12:09 ` Zhang, Tianfei
2023-01-19 13:33 ` Greg KH
2023-01-19 13:43 ` Rafael J. Wysocki
2023-01-19 15:33 ` Greg KH [this message]
2023-01-20 16:28 ` Russ Weight
2023-01-20 18:42 ` Lukas Wunner
2023-01-21 7:34 ` Greg KH
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y8lisT7xmkd1/g+J@kroah.com \
--to=gregkh@linuxfoundation.org \
--cc=andriy.shevchenko@linux.intel.com \
--cc=bhelgaas@google.com \
--cc=corbet@lwn.net \
--cc=dan.j.williams@intel.com \
--cc=hao.wu@intel.com \
--cc=ilpo.jarvinen@linux.intel.com \
--cc=ira.weiny@intel.com \
--cc=jgg@ziepe.ca \
--cc=kabel@kernel.org \
--cc=keescook@chromium.org \
--cc=lee@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fpga@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lukas@wunner.de \
--cc=mani@kernel.org \
--cc=matthew.gerlach@linux.intel.com \
--cc=mdf@kernel.org \
--cc=pali@kernel.org \
--cc=rafael@kernel.org \
--cc=russell.h.weight@intel.com \
--cc=tianfei.zhang@intel.com \
--cc=trix@redhat.com \
--cc=yilun.xu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).