* vfio with iommu groups
@ 2012-05-12 7:31 ` Alexey Kardashevskiy
0 siblings, 0 replies; 6+ messages in thread
From: Alexey Kardashevskiy @ 2012-05-12 7:31 UTC (permalink / raw)
To: Alex Williamson, Alex Williamson, Benjamin Herrenschmidt,
David Gibson, anthony, Alex Graf, kvm, qemu-devel
Hi!
I pulled new VFIO from github, ported to POWER and got some issues/thoughts which I post as patches.
However PCI bridges handling is an open question to discuss.
My test setup includes PCIe card Intel E1000E which looks in the host like this
(cut device names in the tree below as they were too long to look nice):
aik@vpl2:~$ lspci -vt
+-[0003:00]---00.0-[01-ff]----00.0-[02-04]--+-02.0-[03]--+-00.0 Intel Gigabit Ethernet
| | \-00.1 Intel Gigabit Ethernet
| \-04.0-[04]--+-00.0 Intel Gigabit Ethernet
| \-00.1 Intel Gigabit Ethernet
aik@vpl2:~$ lspci
...
0003:00:00.0 PCI bridge: IBM Device 02ea (rev 02)
0003:01:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
0003:02:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
0003:02:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
0003:03:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
0003:03:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
0003:04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
0003:04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
At the moment I bind all devices from domain 0003 to the vfio_pci driver and give as many
devices from it to QEMU as I want. vfio-pci owns devices, the guest sees what he needs to see -
ethernet adapters, it works.
However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
For example, INTx lines should be swizzled when the guest parses a device tree and
tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
functions.
Is there any idea what to do with bridges?
--
Alexey
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Qemu-devel] vfio with iommu groups
@ 2012-05-12 7:31 ` Alexey Kardashevskiy
0 siblings, 0 replies; 6+ messages in thread
From: Alexey Kardashevskiy @ 2012-05-12 7:31 UTC (permalink / raw)
To: Alex Williamson
Hi!
I pulled new VFIO from github, ported to POWER and got some issues/thoughts which I post as patches.
However PCI bridges handling is an open question to discuss.
My test setup includes PCIe card Intel E1000E which looks in the host like this
(cut device names in the tree below as they were too long to look nice):
aik@vpl2:~$ lspci -vt
+-[0003:00]---00.0-[01-ff]----00.0-[02-04]--+-02.0-[03]--+-00.0 Intel Gigabit Ethernet
| | \-00.1 Intel Gigabit Ethernet
| \-04.0-[04]--+-00.0 Intel Gigabit Ethernet
| \-00.1 Intel Gigabit Ethernet
aik@vpl2:~$ lspci
...
0003:00:00.0 PCI bridge: IBM Device 02ea (rev 02)
0003:01:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
0003:02:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
0003:02:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
0003:03:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
0003:03:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
0003:04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
0003:04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
(rev 06)
At the moment I bind all devices from domain 0003 to the vfio_pci driver and give as many
devices from it to QEMU as I want. vfio-pci owns devices, the guest sees what he needs to see -
ethernet adapters, it works.
However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
For example, INTx lines should be swizzled when the guest parses a device tree and
tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
functions.
Is there any idea what to do with bridges?
--
Alexey
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: vfio with iommu groups
2012-05-12 7:31 ` [Qemu-devel] " Alexey Kardashevskiy
@ 2012-05-14 4:16 ` Alex Williamson
-1 siblings, 0 replies; 6+ messages in thread
From: Alex Williamson @ 2012-05-14 4:16 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: kvm, qemu-devel, Alex Graf, anthony, David Gibson
On Sat, 2012-05-12 at 17:31 +1000, Alexey Kardashevskiy wrote:
> Hi!
>
> I pulled new VFIO from github, ported to POWER and got some issues/thoughts which I post as patches.
> However PCI bridges handling is an open question to discuss.
>
> My test setup includes PCIe card Intel E1000E which looks in the host like this
> (cut device names in the tree below as they were too long to look nice):
> aik@vpl2:~$ lspci -vt
> +-[0003:00]---00.0-[01-ff]----00.0-[02-04]--+-02.0-[03]--+-00.0 Intel Gigabit Ethernet
> | | \-00.1 Intel Gigabit Ethernet
> | \-04.0-[04]--+-00.0 Intel Gigabit Ethernet
> | \-00.1 Intel Gigabit Ethernet
>
> aik@vpl2:~$ lspci
> ...
> 0003:00:00.0 PCI bridge: IBM Device 02ea (rev 02)
> 0003:01:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:02:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:02:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:03:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:03:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
>
>
> At the moment I bind all devices from domain 0003 to the vfio_pci driver and give as many
> devices from it to QEMU as I want. vfio-pci owns devices, the guest sees what he needs to see -
> ethernet adapters, it works.
Yay!
> However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
> For example, INTx lines should be swizzled when the guest parses a device tree and
> tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
> functions.
>
> Is there any idea what to do with bridges?
I don't really have a problem with the idea of exposing bridges, but it
get's pretty complicated. What happens if the guest reprograms the
subordinate bus numbers? Is there any advantage vs exposing an emulated
bridge in it's place? Do you have a proposal for it? Thanks,
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] vfio with iommu groups
@ 2012-05-14 4:16 ` Alex Williamson
0 siblings, 0 replies; 6+ messages in thread
From: Alex Williamson @ 2012-05-14 4:16 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: kvm, qemu-devel, Alex Graf, anthony, David Gibson
On Sat, 2012-05-12 at 17:31 +1000, Alexey Kardashevskiy wrote:
> Hi!
>
> I pulled new VFIO from github, ported to POWER and got some issues/thoughts which I post as patches.
> However PCI bridges handling is an open question to discuss.
>
> My test setup includes PCIe card Intel E1000E which looks in the host like this
> (cut device names in the tree below as they were too long to look nice):
> aik@vpl2:~$ lspci -vt
> +-[0003:00]---00.0-[01-ff]----00.0-[02-04]--+-02.0-[03]--+-00.0 Intel Gigabit Ethernet
> | | \-00.1 Intel Gigabit Ethernet
> | \-04.0-[04]--+-00.0 Intel Gigabit Ethernet
> | \-00.1 Intel Gigabit Ethernet
>
> aik@vpl2:~$ lspci
> ...
> 0003:00:00.0 PCI bridge: IBM Device 02ea (rev 02)
> 0003:01:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:02:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:02:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e)
> 0003:03:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:03:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
> 0003:04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
> (rev 06)
>
>
> At the moment I bind all devices from domain 0003 to the vfio_pci driver and give as many
> devices from it to QEMU as I want. vfio-pci owns devices, the guest sees what he needs to see -
> ethernet adapters, it works.
Yay!
> However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
> For example, INTx lines should be swizzled when the guest parses a device tree and
> tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
> functions.
>
> Is there any idea what to do with bridges?
I don't really have a problem with the idea of exposing bridges, but it
get's pretty complicated. What happens if the guest reprograms the
subordinate bus numbers? Is there any advantage vs exposing an emulated
bridge in it's place? Do you have a proposal for it? Thanks,
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: vfio with iommu groups
2012-05-14 4:16 ` [Qemu-devel] " Alex Williamson
@ 2012-05-14 4:29 ` Benjamin Herrenschmidt
-1 siblings, 0 replies; 6+ messages in thread
From: Benjamin Herrenschmidt @ 2012-05-14 4:29 UTC (permalink / raw)
To: Alex Williamson
Cc: kvm, Alexey Kardashevskiy, qemu-devel, Alex Graf, anthony, David Gibson
On Sun, 2012-05-13 at 22:16 -0600, Alex Williamson wrote:
> > However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
> > For example, INTx lines should be swizzled when the guest parses a device tree and
> > tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
> > functions.
> >
> > Is there any idea what to do with bridges?
>
> I don't really have a problem with the idea of exposing bridges, but it
> get's pretty complicated. What happens if the guest reprograms the
> subordinate bus numbers? Is there any advantage vs exposing an emulated
> bridge in it's place? Do you have a proposal for it? Thanks,
I think exposing an emulated bridge would be more in the "spirit" of the
current vfio and safer for now (long run, as we already discussed, we
might look at an other option with more direct HW access for power).
We do want to expose those bridges one way or another for the sake of
getting the interrupt routing right, since the guest will make
assumptions based on those bridges to calculate the proper swizzling.
Cheers,
Ben.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] vfio with iommu groups
@ 2012-05-14 4:29 ` Benjamin Herrenschmidt
0 siblings, 0 replies; 6+ messages in thread
From: Benjamin Herrenschmidt @ 2012-05-14 4:29 UTC (permalink / raw)
To: Alex Williamson
Cc: kvm, Alexey Kardashevskiy, qemu-devel, Alex Graf, anthony, David Gibson
On Sun, 2012-05-13 at 22:16 -0600, Alex Williamson wrote:
> > However theoretically we might want to show these 3 PCIe bridges as well (but not the root complex).
> > For example, INTx lines should be swizzled when the guest parses a device tree and
> > tries to calculate a real IRQ number. VFIO does not handles bridges at all, it treats them as PCI
> > functions.
> >
> > Is there any idea what to do with bridges?
>
> I don't really have a problem with the idea of exposing bridges, but it
> get's pretty complicated. What happens if the guest reprograms the
> subordinate bus numbers? Is there any advantage vs exposing an emulated
> bridge in it's place? Do you have a proposal for it? Thanks,
I think exposing an emulated bridge would be more in the "spirit" of the
current vfio and safer for now (long run, as we already discussed, we
might look at an other option with more direct HW access for power).
We do want to expose those bridges one way or another for the sake of
getting the interrupt routing right, since the guest will make
assumptions based on those bridges to calculate the proper swizzling.
Cheers,
Ben.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-05-14 4:29 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-05-12 7:31 vfio with iommu groups Alexey Kardashevskiy
2012-05-12 7:31 ` [Qemu-devel] " Alexey Kardashevskiy
2012-05-14 4:16 ` Alex Williamson
2012-05-14 4:16 ` [Qemu-devel] " Alex Williamson
2012-05-14 4:29 ` Benjamin Herrenschmidt
2012-05-14 4:29 ` [Qemu-devel] " Benjamin Herrenschmidt
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.