time periods of no resource re-assignment

time periods of no resource re-assignment

[Jan Beulich]

Bjorn,

in the context of dealing with the final part of a long known security
issue on Xen, I'd appreciate if you could help me understand
under what conditions (or at which times) MMIO resources for PCI
devices can get re-assigned in Linux. Or really I'm in need to know
under what conditions there won't be any resource re-assignment
(i.e. we particularly need the BARs to be stable that cover the
MSI-X table and the Pending Bit Array).

Right now I'm intending to send the hypervisor a respective
notification in the context of xen-pciback's probe function, after
having called pci_enable_device(). Can you confirm that at this
point, and until the driver lets go of the device again, resources
won't get re-assigned? Or if not, can you suggest a better point
in time for sending such a notification, or additional steps we
may need to take in order to ensure resource stability?

Thanks a lot, Jan

Re: time periods of no resource re-assignment

[Bjorn Helgaas]

On Thu, Feb 28, 2013 at 2:08 AM, Jan Beulich <JBeulich@suse.com> wrote:
> Bjorn,
>
> in the context of dealing with the final part of a long known security
> issue on Xen, I'd appreciate if you could help me understand
> under what conditions (or at which times) MMIO resources for PCI
> devices can get re-assigned in Linux. Or really I'm in need to know
> under what conditions there won't be any resource re-assignment
> (i.e. we particularly need the BARs to be stable that cover the
> MSI-X table and the Pending Bit Array).
>
> Right now I'm intending to send the hypervisor a respective
> notification in the context of xen-pciback's probe function, after
> having called pci_enable_device(). Can you confirm that at this
> point, and until the driver lets go of the device again, resources
> won't get re-assigned? Or if not, can you suggest a better point
> in time for sending such a notification, or additional steps we
> may need to take in order to ensure resource stability?

Hi Jan,

There is currently no mechanism for the PCI core to tell a driver that
we're changing any device resources (IRQ, MMIO, I/O).  Since there's
no way to tell a driver to refresh its idea of device resources, the
core can not change anything after we call a driver's .probe()
method.

This could change in the future if we support dynamic resource
reassignment, but I think that would require adding some sort of
notification infrastructure so we could essentially suspend a driver,
reassign device resources, and tell the driver "things might have
changed" when we resume it.

Bjorn

Re: time periods of no resource re-assignment

[Bjorn Helgaas]

On Thu, Feb 28, 2013 at 2:08 AM, Jan Beulich <JBeulich@suse.com> wrote:
> Bjorn,
>
> in the context of dealing with the final part of a long known security
> issue on Xen, I'd appreciate if you could help me understand
> under what conditions (or at which times) MMIO resources for PCI
> devices can get re-assigned in Linux. Or really I'm in need to know
> under what conditions there won't be any resource re-assignment
> (i.e. we particularly need the BARs to be stable that cover the
> MSI-X table and the Pending Bit Array).
>
> Right now I'm intending to send the hypervisor a respective
> notification in the context of xen-pciback's probe function, after
> having called pci_enable_device(). Can you confirm that at this
> point, and until the driver lets go of the device again, resources
> won't get re-assigned? Or if not, can you suggest a better point
> in time for sending such a notification, or additional steps we
> may need to take in order to ensure resource stability?

Hi Jan,

There is currently no mechanism for the PCI core to tell a driver that
we're changing any device resources (IRQ, MMIO, I/O).  Since there's
no way to tell a driver to refresh its idea of device resources, the
core can not change anything after we call a driver's .probe()
method.

This could change in the future if we support dynamic resource
reassignment, but I think that would require adding some sort of
notification infrastructure so we could essentially suspend a driver,
reassign device resources, and tell the driver "things might have
changed" when we resume it.

Bjorn

Re: time periods of no resource re-assignment

[Jan Beulich]

>>> On 28.02.13 at 17:06, Bjorn Helgaas <bhelgaas@google.com> wrote:
> There is currently no mechanism for the PCI core to tell a driver that
> we're changing any device resources (IRQ, MMIO, I/O).  Since there's
> no way to tell a driver to refresh its idea of device resources, the
> core can not change anything after we call a driver's .probe()
> method.
> 
> This could change in the future if we support dynamic resource
> reassignment, but I think that would require adding some sort of
> notification infrastructure so we could essentially suspend a driver,
> reassign device resources, and tell the driver "things might have
> changed" when we resume it.

Thanks Bjorn! That should allow us to get things working for now
the way I have coded them up already, and I assume eventual
changes in this area (as you outlined) would imply cooperation by
the driver (i.e. if a device is assigned to a guest, it ought to be
possible to deny the re-assignment attempt if there's no way to
let the guest know of the change).

Jan

Re: time periods of no resource re-assignment

[Jan Beulich]

>>> On 28.02.13 at 17:06, Bjorn Helgaas <bhelgaas@google.com> wrote:
> There is currently no mechanism for the PCI core to tell a driver that
> we're changing any device resources (IRQ, MMIO, I/O).  Since there's
> no way to tell a driver to refresh its idea of device resources, the
> core can not change anything after we call a driver's .probe()
> method.
> 
> This could change in the future if we support dynamic resource
> reassignment, but I think that would require adding some sort of
> notification infrastructure so we could essentially suspend a driver,
> reassign device resources, and tell the driver "things might have
> changed" when we resume it.

Thanks Bjorn! That should allow us to get things working for now
the way I have coded them up already, and I assume eventual
changes in this area (as you outlined) would imply cooperation by
the driver (i.e. if a device is assigned to a guest, it ought to be
possible to deny the re-assignment attempt if there's no way to
let the guest know of the change).

Jan

time periods of no resource re-assignment

[Jan Beulich]

Bjorn,

in the context of dealing with the final part of a long known security
issue on Xen, I'd appreciate if you could help me understand
under what conditions (or at which times) MMIO resources for PCI
devices can get re-assigned in Linux. Or really I'm in need to know
under what conditions there won't be any resource re-assignment
(i.e. we particularly need the BARs to be stable that cover the
MSI-X table and the Pending Bit Array).

Right now I'm intending to send the hypervisor a respective
notification in the context of xen-pciback's probe function, after
having called pci_enable_device(). Can you confirm that at this
point, and until the driver lets go of the device again, resources
won't get re-assigned? Or if not, can you suggest a better point
in time for sending such a notification, or additional steps we
may need to take in order to ensure resource stability?

Thanks a lot, Jan