FLR support in xl tool stack

FLR support in xl tool stack

[Kay, Allen M]

In xm tool stack, there is a function called do_FLR() located in python/xen/util/pci.py that takes care of issuing FLR to passthrough PCI devices.  This function is responsible for looking at the device and figure out the most appropriate way to reset the device (PCI/PCIe FLR, secondary bus reset, etc...).

In xl tool stack, I see there is a libxl_device_pci_reset() in libxl/libxl_pci.c that suppose to do the similar thing.  It references /sys/bus/pci/drivers/pciback/do_flr and tries to write something to it.  However, pciback driver in 2.6.32.27 pvops kernel does not seem to have this do_flr functionality.

Is this something being worked on but not checked in to pvops dom0 kernel yet?

If not, I would prefer implementing all of the do_FLR() functionaility in libxl and not to depend on pciback driver given it is not part of the upstream kernel.  This will make using newer kernels such as 2.6.38 much easier.

Comments?

Allen

Re: FLR support in xl tool stack

[Jean Guyader]

On 28 January 2011 08:38, Kay, Allen M <allen.m.kay@intel.com> wrote:
> In xm tool stack, there is a function called do_FLR() located in python/xen/util/pci.py that takes care of issuing FLR to passthrough PCI devices.  This function is responsible for looking at the device and figure out the most appropriate way to reset the device (PCI/PCIe FLR, secondary bus reset, etc...).
>
> In xl tool stack, I see there is a libxl_device_pci_reset() in libxl/libxl_pci.c that suppose to do the similar thing.  It references /sys/bus/pci/drivers/pciback/do_flr and tries to write something to it.  However, pciback driver in 2.6.32.27 pvops kernel does not seem to have this do_flr functionality.
>
> Is this something being worked on but not checked in to pvops dom0 kernel yet?
>
> If not, I would prefer implementing all of the do_FLR() functionaility in libxl and not to depend on pciback driver given it is not part of the upstream kernel.  This will make using newer kernels such as 2.6.38 much easier.
>
> Comments?
>

On the newest kernels (I don't know from which version), the reset
function has been moved under the pci device tree directly. So
/sys/bus/pci/device/<BDF>/reset.

Jean

Re: FLR support in xl tool stack

[Ian Campbell]

On Fri, 2011-01-28 at 04:20 +0000, Jean Guyader wrote: 
> On 28 January 2011 08:38, Kay, Allen M <allen.m.kay@intel.com> wrote:
> > In xm tool stack, there is a function called do_FLR() located in
> python/xen/util/pci.py that takes care of issuing FLR to passthrough
> PCI devices.  This function is responsible for looking at the device
> and figure out the most appropriate way to reset the device (PCI/PCIe
> FLR, secondary bus reset, etc...).
> >
> > In xl tool stack, I see there is a libxl_device_pci_reset() in
> libxl/libxl_pci.c that suppose to do the similar thing.  It
> references /sys/bus/pci/drivers/pciback/do_flr and tries to write
> something to it.  However, pciback driver in 2.6.32.27 pvops kernel
> does not seem to have this do_flr functionality.
> >
> > Is this something being worked on but not checked in to pvops dom0
> kernel yet?
> >
> > If not, I would prefer implementing all of the do_FLR()
> functionaility in libxl and not to depend on pciback driver given it
> is not part of the upstream kernel.  This will make using newer
> kernels such as 2.6.38 much easier. 
> >
> > Comments?
> >
> 
> On the newest kernels (I don't know from which version), the reset
> function has been moved under the pci device tree directly. So
> /sys/bus/pci/device/<BDF>/reset.
Only...             ^s

... otherwise that's correct and libxl will try this path if the pciback
one does not exist.

If any additional FLR functionality is required it should be done in the
kernel as part of this generic FLR mechanism so that everyone benefits.

drivers/pci/pci-sysfs.c provides the sysfs node based on the result of
pci_probe_reset_function(dev) which tries a bunch of generic ways of
resetting a device. In addition drivers/pci/quirks.c provides some
device specific quirks.

Perhaps all that is needed is to nominate some patches for backporting
to the xen/stable-2.6.32 branch? 
$ git log --no-merges --pretty=oneline v2.6.32.27..origin/master drivers/pci/quirks.c
seems to provide an initial set of potentially interesting candidates.

Ian.

RE: FLR support in xl tool stack

[Ross Philipson]

Yeah when I first implemented this in pciback is was looking for a common place since the existing FLR logic was in the xend tool-stack (which we were not using). Also folks on xen-devel had suggested it go in pciback at the time.

I am not familiar with what reset functionality is in pci in newer kernels (there was not a lot in the .27 one) but a lot of what I put in pciback was specific to resetting the Intel graphics device on GM45 and Q45 chipsets.

Thanks
Ross

-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@gmail.com] 
Sent: Thursday, January 27, 2011 11:20 PM
To: Kay, Allen M
Cc: xen-devel; Wilk; Stefano Stabellini; Jean Guyader; Ian Campbell; Thomas Horsten; Ross Philipson
Subject: Re: [Xen-devel] FLR support in xl tool stack

On 28 January 2011 08:38, Kay, Allen M <allen.m.kay@intel.com> wrote:
> In xm tool stack, there is a function called do_FLR() located in python/xen/util/pci.py that takes care of issuing FLR to passthrough PCI devices.  This function is responsible for looking at the device and figure out the most appropriate way to reset the device (PCI/PCIe FLR, secondary bus reset, etc...).
>
> In xl tool stack, I see there is a libxl_device_pci_reset() in libxl/libxl_pci.c that suppose to do the similar thing.  It references /sys/bus/pci/drivers/pciback/do_flr and tries to write something to it.  However, pciback driver in 2.6.32.27 pvops kernel does not seem to have this do_flr functionality.
>
> Is this something being worked on but not checked in to pvops dom0 kernel yet?
>
> If not, I would prefer implementing all of the do_FLR() functionaility in libxl and not to depend on pciback driver given it is not part of the upstream kernel.  This will make using newer kernels such as 2.6.38 much easier.
>
> Comments?
>

On the newest kernels (I don't know from which version), the reset function has been moved under the pci device tree directly. So /sys/bus/pci/device/<BDF>/reset.

Jean