[Qemu-devel] [RFC] pxb: Restrict to x86

[Qemu-devel] [RFC] pxb: Restrict to x86

[David Gibson]

The PCI Expander Bridge (PXB) device is essentially a hack to allow
different PCIe devices to be assigned to different NUMA nodes on x86.  Each
PXB is sort-of a separate PCI host bridge, except that its config space
is shared with the config space of the main PCI host bridge, rather than
being independent.

This is only necessary if the platform doesn't (easily) allow truly
independent PCI host bridges.  AFAIK that's just x86.

This patch makes it possible to configure PXB out of the build, and adjusts
the default configs so it's only included on x86 targets.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 default-configs/i386-softmmu.mak   | 1 +
 default-configs/x86_64-softmmu.mak | 1 +
 hw/pci-bridge/Makefile.objs        | 2 +-
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
index 0b51360..6c52d26 100644
--- a/default-configs/i386-softmmu.mak
+++ b/default-configs/i386-softmmu.mak
@@ -56,3 +56,4 @@ CONFIG_IOH3420=y
 CONFIG_I82801B11=y
 CONFIG_SMBIOS=y
 CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
+CONFIG_PXB=y
diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
index 7f89503..2d0341c 100644
--- a/default-configs/x86_64-softmmu.mak
+++ b/default-configs/x86_64-softmmu.mak
@@ -56,3 +56,4 @@ CONFIG_IOH3420=y
 CONFIG_I82801B11=y
 CONFIG_SMBIOS=y
 CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
+CONFIG_PXB=y
diff --git a/hw/pci-bridge/Makefile.objs b/hw/pci-bridge/Makefile.objs
index f2adfe3..5612bd7 100644
--- a/hw/pci-bridge/Makefile.objs
+++ b/hw/pci-bridge/Makefile.objs
@@ -1,5 +1,5 @@
 common-obj-y += pci_bridge_dev.o
-common-obj-y += pci_expander_bridge.o
+common-obj-$(CONFIG_PXB) += pci_expander_bridge.o
 common-obj-$(CONFIG_XIO3130) += xio3130_upstream.o xio3130_downstream.o
 common-obj-$(CONFIG_IOH3420) += ioh3420.o
 common-obj-$(CONFIG_I82801B11) += i82801b11.o
-- 
2.9.3

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[Marcel Apfelbaum]

On 01/06/2017 07:04 AM, David Gibson wrote:
> The PCI Expander Bridge (PXB) device is essentially a hack to allow
> different PCIe devices to be assigned to different NUMA nodes on x86.  Each
> PXB is sort-of a separate PCI host bridge, except that its config space
> is shared with the config space of the main PCI host bridge, rather than
> being independent.
>

Hi David,

> This is only necessary if the platform doesn't (easily) allow truly
> independent PCI host bridges.  AFAIK that's just x86.
>

Indeed, it is possible to support independent PCI host bridges on x86 by
using a separate MMCONFIG space for each one and enable separate PCI domains.
We simply didn't need this until now, but maybe will be implemented it in the future.

> This patch makes it possible to configure PXB out of the build, and adjusts
> the default configs so it's only included on x86 targets.
>

Makes sense

Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>

Thanks,
Marcel

> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  default-configs/i386-softmmu.mak   | 1 +
>  default-configs/x86_64-softmmu.mak | 1 +
>  hw/pci-bridge/Makefile.objs        | 2 +-
>  3 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
> index 0b51360..6c52d26 100644
> --- a/default-configs/i386-softmmu.mak
> +++ b/default-configs/i386-softmmu.mak
> @@ -56,3 +56,4 @@ CONFIG_IOH3420=y
>  CONFIG_I82801B11=y
>  CONFIG_SMBIOS=y
>  CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
> +CONFIG_PXB=y
> diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
> index 7f89503..2d0341c 100644
> --- a/default-configs/x86_64-softmmu.mak
> +++ b/default-configs/x86_64-softmmu.mak
> @@ -56,3 +56,4 @@ CONFIG_IOH3420=y
>  CONFIG_I82801B11=y
>  CONFIG_SMBIOS=y
>  CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
> +CONFIG_PXB=y
> diff --git a/hw/pci-bridge/Makefile.objs b/hw/pci-bridge/Makefile.objs
> index f2adfe3..5612bd7 100644
> --- a/hw/pci-bridge/Makefile.objs
> +++ b/hw/pci-bridge/Makefile.objs
> @@ -1,5 +1,5 @@
>  common-obj-y += pci_bridge_dev.o
> -common-obj-y += pci_expander_bridge.o
> +common-obj-$(CONFIG_PXB) += pci_expander_bridge.o
>  common-obj-$(CONFIG_XIO3130) += xio3130_upstream.o xio3130_downstream.o
>  common-obj-$(CONFIG_IOH3420) += ioh3420.o
>  common-obj-$(CONFIG_I82801B11) += i82801b11.o
>

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[Marcel Apfelbaum]

On 01/06/2017 08:13 AM, Marcel Apfelbaum wrote:
> On 01/06/2017 07:04 AM, David Gibson wrote:
>> The PCI Expander Bridge (PXB) device is essentially a hack to allow
>> different PCIe devices to be assigned to different NUMA nodes on x86.  Each
>> PXB is sort-of a separate PCI host bridge, except that its config space
>> is shared with the config space of the main PCI host bridge, rather than
>> being independent.
>>
>
> Hi David,
>
>> This is only necessary if the platform doesn't (easily) allow truly
>> independent PCI host bridges.  AFAIK that's just x86.
>>
>
> Indeed, it is possible to support independent PCI host bridges on x86 by
> using a separate MMCONFIG space for each one and enable separate PCI domains.
> We simply didn't need this until now, but maybe will be implemented it in the future.
>
>> This patch makes it possible to configure PXB out of the build, and adjusts
>> the default configs so it's only included on x86 targets.
>>
>
> Makes sense

Actually ACPI based ARM guests may be able to use the PXB as well,
let's wait for their opinion.

Thanks,
Marcel

>
> Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
>
> Thanks,
> Marcel
>
>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
>> ---
>>  default-configs/i386-softmmu.mak   | 1 +
>>  default-configs/x86_64-softmmu.mak | 1 +
>>  hw/pci-bridge/Makefile.objs        | 2 +-
>>  3 files changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
>> index 0b51360..6c52d26 100644
>> --- a/default-configs/i386-softmmu.mak
>> +++ b/default-configs/i386-softmmu.mak
>> @@ -56,3 +56,4 @@ CONFIG_IOH3420=y
>>  CONFIG_I82801B11=y
>>  CONFIG_SMBIOS=y
>>  CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
>> +CONFIG_PXB=y
>> diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
>> index 7f89503..2d0341c 100644
>> --- a/default-configs/x86_64-softmmu.mak
>> +++ b/default-configs/x86_64-softmmu.mak
>> @@ -56,3 +56,4 @@ CONFIG_IOH3420=y
>>  CONFIG_I82801B11=y
>>  CONFIG_SMBIOS=y
>>  CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
>> +CONFIG_PXB=y
>> diff --git a/hw/pci-bridge/Makefile.objs b/hw/pci-bridge/Makefile.objs
>> index f2adfe3..5612bd7 100644
>> --- a/hw/pci-bridge/Makefile.objs
>> +++ b/hw/pci-bridge/Makefile.objs
>> @@ -1,5 +1,5 @@
>>  common-obj-y += pci_bridge_dev.o
>> -common-obj-y += pci_expander_bridge.o
>> +common-obj-$(CONFIG_PXB) += pci_expander_bridge.o
>>  common-obj-$(CONFIG_XIO3130) += xio3130_upstream.o xio3130_downstream.o
>>  common-obj-$(CONFIG_IOH3420) += ioh3420.o
>>  common-obj-$(CONFIG_I82801B11) += i82801b11.o
>>
>
>

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[Michael S. Tsirkin]

On Fri, Jan 06, 2017 at 08:13:11AM +0200, Marcel Apfelbaum wrote:
> On 01/06/2017 07:04 AM, David Gibson wrote:
> > The PCI Expander Bridge (PXB) device is essentially a hack to allow
> > different PCIe devices to be assigned to different NUMA nodes on x86.  Each
> > PXB is sort-of a separate PCI host bridge, except that its config space
> > is shared with the config space of the main PCI host bridge, rather than
> > being independent.
> > 
> 
> Hi David,
> 
> > This is only necessary if the platform doesn't (easily) allow truly
> > independent PCI host bridges.  AFAIK that's just x86.
> > 
> 
> Indeed, it is possible to support independent PCI host bridges on x86 by
> using a separate MMCONFIG space for each one and enable separate PCI domains.
> We simply didn't need this until now, but maybe will be implemented it in the future.

In fact I would say that's the cleanest way to do this on q35.

> > This patch makes it possible to configure PXB out of the build, and adjusts
> > the default configs so it's only included on x86 targets.
> > 
> 
> Makes sense
> 
> Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
> 
> Thanks,
> Marcel
> 
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > ---
> >  default-configs/i386-softmmu.mak   | 1 +
> >  default-configs/x86_64-softmmu.mak | 1 +
> >  hw/pci-bridge/Makefile.objs        | 2 +-
> >  3 files changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
> > index 0b51360..6c52d26 100644
> > --- a/default-configs/i386-softmmu.mak
> > +++ b/default-configs/i386-softmmu.mak
> > @@ -56,3 +56,4 @@ CONFIG_IOH3420=y
> >  CONFIG_I82801B11=y
> >  CONFIG_SMBIOS=y
> >  CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
> > +CONFIG_PXB=y
> > diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
> > index 7f89503..2d0341c 100644
> > --- a/default-configs/x86_64-softmmu.mak
> > +++ b/default-configs/x86_64-softmmu.mak
> > @@ -56,3 +56,4 @@ CONFIG_IOH3420=y
> >  CONFIG_I82801B11=y
> >  CONFIG_SMBIOS=y
> >  CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM)
> > +CONFIG_PXB=y
> > diff --git a/hw/pci-bridge/Makefile.objs b/hw/pci-bridge/Makefile.objs
> > index f2adfe3..5612bd7 100644
> > --- a/hw/pci-bridge/Makefile.objs
> > +++ b/hw/pci-bridge/Makefile.objs
> > @@ -1,5 +1,5 @@
> >  common-obj-y += pci_bridge_dev.o
> > -common-obj-y += pci_expander_bridge.o
> > +common-obj-$(CONFIG_PXB) += pci_expander_bridge.o
> >  common-obj-$(CONFIG_XIO3130) += xio3130_upstream.o xio3130_downstream.o
> >  common-obj-$(CONFIG_IOH3420) += ioh3420.o
> >  common-obj-$(CONFIG_I82801B11) += i82801b11.o
> > 

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[Marcel Apfelbaum]

On 01/06/2017 09:41 PM, Michael S. Tsirkin wrote:
> On Fri, Jan 06, 2017 at 08:13:11AM +0200, Marcel Apfelbaum wrote:
>> On 01/06/2017 07:04 AM, David Gibson wrote:
>>> The PCI Expander Bridge (PXB) device is essentially a hack to allow
>>> different PCIe devices to be assigned to different NUMA nodes on x86.  Each
>>> PXB is sort-of a separate PCI host bridge, except that its config space
>>> is shared with the config space of the main PCI host bridge, rather than
>>> being independent.
>>>
>>
>> Hi David,
>>
>>> This is only necessary if the platform doesn't (easily) allow truly
>>> independent PCI host bridges.  AFAIK that's just x86.
>>>
>>
>> Indeed, it is possible to support independent PCI host bridges on x86 by
>> using a separate MMCONFIG space for each one and enable separate PCI domains.
>> We simply didn't need this until now, but maybe will be implemented it in the future.
>
> In fact I would say that's the cleanest way to do this on q35.
>

Message received :)

Thanks,
Marcel


>>> This patch makes it possible to configure PXB out of the build, and adjusts
>>> the default configs so it's only included on x86 targets.
>>>
>>

[...]

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

On Sun, Jan 08, 2017 at 10:17:12AM +0200, Marcel Apfelbaum wrote:
> On 01/06/2017 09:41 PM, Michael S. Tsirkin wrote:
> > On Fri, Jan 06, 2017 at 08:13:11AM +0200, Marcel Apfelbaum wrote:
> > > On 01/06/2017 07:04 AM, David Gibson wrote:
> > > > The PCI Expander Bridge (PXB) device is essentially a hack to allow
> > > > different PCIe devices to be assigned to different NUMA nodes on x86.  Each
> > > > PXB is sort-of a separate PCI host bridge, except that its config space
> > > > is shared with the config space of the main PCI host bridge, rather than
> > > > being independent.
> > > > 
> > > 
> > > Hi David,
> > > 
> > > > This is only necessary if the platform doesn't (easily) allow truly
> > > > independent PCI host bridges.  AFAIK that's just x86.
> > > > 
> > > 
> > > Indeed, it is possible to support independent PCI host bridges on x86 by
> > > using a separate MMCONFIG space for each one and enable separate PCI domains.
> > > We simply didn't need this until now, but maybe will be implemented it in the future.
> > 
> > In fact I would say that's the cleanest way to do this on q35.
> 
> Message received :)

Just so I'm clear, these last two comments are essentially suggesting
a follow up cleanup on x86, rather than suggesting a different
approach for non-PC platforms, yes?

If there are no objections to my original patch do you want to take it
through your tree Michael, or should I take it through mine?

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[Michael S. Tsirkin]

On Mon, Jan 09, 2017 at 10:48:24AM +1100, David Gibson wrote:
> On Sun, Jan 08, 2017 at 10:17:12AM +0200, Marcel Apfelbaum wrote:
> > On 01/06/2017 09:41 PM, Michael S. Tsirkin wrote:
> > > On Fri, Jan 06, 2017 at 08:13:11AM +0200, Marcel Apfelbaum wrote:
> > > > On 01/06/2017 07:04 AM, David Gibson wrote:
> > > > > The PCI Expander Bridge (PXB) device is essentially a hack to allow
> > > > > different PCIe devices to be assigned to different NUMA nodes on x86.  Each
> > > > > PXB is sort-of a separate PCI host bridge, except that its config space
> > > > > is shared with the config space of the main PCI host bridge, rather than
> > > > > being independent.
> > > > > 
> > > > 
> > > > Hi David,
> > > > 
> > > > > This is only necessary if the platform doesn't (easily) allow truly
> > > > > independent PCI host bridges.  AFAIK that's just x86.
> > > > > 
> > > > 
> > > > Indeed, it is possible to support independent PCI host bridges on x86 by
> > > > using a separate MMCONFIG space for each one and enable separate PCI domains.
> > > > We simply didn't need this until now, but maybe will be implemented it in the future.
> > > 
> > > In fact I would say that's the cleanest way to do this on q35.
> > 
> > Message received :)
> 
> Just so I'm clear, these last two comments are essentially suggesting
> a follow up cleanup on x86, rather than suggesting a different
> approach for non-PC platforms, yes?
> 
> If there are no objections to my original patch do you want to take it
> through your tree Michael, or should I take it through mine?
> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson



Go ahead and take it through  yours pls.

Acked-by: Michael S. Tsirkin <mst@redhat.com>

Re: [Qemu-devel] [RFC] pxb: Restrict to x86

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

On Mon, Jan 09, 2017 at 03:58:03AM +0200, Michael S. Tsirkin wrote:
> On Mon, Jan 09, 2017 at 10:48:24AM +1100, David Gibson wrote:
> > On Sun, Jan 08, 2017 at 10:17:12AM +0200, Marcel Apfelbaum wrote:
> > > On 01/06/2017 09:41 PM, Michael S. Tsirkin wrote:
> > > > On Fri, Jan 06, 2017 at 08:13:11AM +0200, Marcel Apfelbaum wrote:
> > > > > On 01/06/2017 07:04 AM, David Gibson wrote:
> > > > > > The PCI Expander Bridge (PXB) device is essentially a hack to allow
> > > > > > different PCIe devices to be assigned to different NUMA nodes on x86.  Each
> > > > > > PXB is sort-of a separate PCI host bridge, except that its config space
> > > > > > is shared with the config space of the main PCI host bridge, rather than
> > > > > > being independent.
> > > > > > 
> > > > > 
> > > > > Hi David,
> > > > > 
> > > > > > This is only necessary if the platform doesn't (easily) allow truly
> > > > > > independent PCI host bridges.  AFAIK that's just x86.
> > > > > > 
> > > > > 
> > > > > Indeed, it is possible to support independent PCI host bridges on x86 by
> > > > > using a separate MMCONFIG space for each one and enable separate PCI domains.
> > > > > We simply didn't need this until now, but maybe will be implemented it in the future.
> > > > 
> > > > In fact I would say that's the cleanest way to do this on q35.
> > > 
> > > Message received :)
> > 
> > Just so I'm clear, these last two comments are essentially suggesting
> > a follow up cleanup on x86, rather than suggesting a different
> > approach for non-PC platforms, yes?
> > 
> > If there are no objections to my original patch do you want to take it
> > through your tree Michael, or should I take it through mine?
> > 
> 
> 
> 
> Go ahead and take it through  yours pls.
> 
> Acked-by: Michael S. Tsirkin <mst@redhat.com>

Done, thanks.


-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]