* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 14:56 ` Anthony PERARD
@ 2014-01-09 15:36 ` Pasi Kärkkäinen
2014-01-09 15:43 ` Pasi Kärkkäinen
2014-01-09 23:54 ` Sander Eikelenboom
` (3 subsequent siblings)
4 siblings, 1 reply; 39+ messages in thread
From: Pasi Kärkkäinen @ 2014-01-09 15:36 UTC (permalink / raw)
To: Anthony PERARD; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
On Thu, Jan 09, 2014 at 02:56:24PM +0000, Anthony PERARD wrote:
> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> > > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
> > > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> > > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> > > > > [...]
> > > > > > > Those Xen report something like:
> > > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
> > > > > > > 131328
> > > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
> > > > > > > memflags=0 (62 of 64)
> > > > > > >
> > > > > > > ?
> > > > > > >
> > > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
> > > > > > > QEMU :) )
> > > > > > >
> > >
> > > > -bash-4.1# lspci -s 01:00.0 -v
> > > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> > > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> > > > Flags: fast devsel, IRQ 16
> > > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
> > > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> > > > I/O ports at e020 [disabled] [size=32]
> > > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
> > > > Expansion ROM at fb400000 [disabled] [size=4M]
> > >
> > > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> > > allocate memory for it. Will have maybe have to find another way.
> > > qemu-trad those not seems to allocate memory, but I haven't been very
> > > far in trying to check that.
> >
> > And indeed that is the case. The "Fix" below fixes it.
> >
> >
> > Based on that and this guest config:
> > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> > memory = 2048
> > boot="d"
> > maxvcpus=32
> > vcpus=1
> > serial='pty'
> > vnclisten="0.0.0.0"
> > name="latest"
> > vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
> > pci = ["01:00.0"]
> >
> > I can boot the guest.
>
> And can you access the ROM from the guest ?
>
>
> Also, I have another patch, it will initialize the PCI ROM BAR like any
> other BAR. In this case, if qemu is envolved in the access to ROM, it
> will print an error, like it the case for other BAR.
>
> I tried to test it, but it was with an embedded VGA card. When I dump
> the ROM, I got the same one as the emulated card instead of the ROM from
> the device.
>
This issue has been reported multiple times on the list, and discussed during the last couple of months on another threads,
mostly related to GPU passthru.
I think some clues were found recently about why you get the emulated rom instead of the actual device rom.
Sorry that I don't have the link available right now..
-- Pasi
>
> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> index 6dd7a68..2bbdb6d 100644
> --- a/hw/xen/xen_pt.c
> +++ b/hw/xen/xen_pt.c
> @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>
> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>
> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> - "xen-pci-pt-rom", d->rom.size);
> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> + "xen-pci-pt-rom", d->rom.size);
> pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> &s->rom);
>
>
> --
> Anthony PERARD
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 15:36 ` Pasi Kärkkäinen
@ 2014-01-09 15:43 ` Pasi Kärkkäinen
0 siblings, 0 replies; 39+ messages in thread
From: Pasi Kärkkäinen @ 2014-01-09 15:43 UTC (permalink / raw)
To: Anthony PERARD; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
On Thu, Jan 09, 2014 at 05:36:40PM +0200, Pasi Kärkkäinen wrote:
> > >
> > > I can boot the guest.
> >
> > And can you access the ROM from the guest ?
> >
> >
> > Also, I have another patch, it will initialize the PCI ROM BAR like any
> > other BAR. In this case, if qemu is envolved in the access to ROM, it
> > will print an error, like it the case for other BAR.
> >
> > I tried to test it, but it was with an embedded VGA card. When I dump
> > the ROM, I got the same one as the emulated card instead of the ROM from
> > the device.
> >
>
> This issue has been reported multiple times on the list, and discussed during the last couple of months on another threads,
> mostly related to GPU passthru.
>
> I think some clues were found recently about why you get the emulated rom instead of the actual device rom.
> Sorry that I don't have the link available right now..
>
Heh.. it seems it was Konrad in this very same thread :)
http://lists.xen.org/archives/html/xen-devel/2013-12/msg02837.html
-- Pasi
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 14:56 ` Anthony PERARD
2014-01-09 15:36 ` Pasi Kärkkäinen
@ 2014-01-09 23:54 ` Sander Eikelenboom
2014-01-10 0:17 ` Sander Eikelenboom
2014-01-10 3:28 ` Konrad Rzeszutek Wilk
` (2 subsequent siblings)
4 siblings, 1 reply; 39+ messages in thread
From: Sander Eikelenboom @ 2014-01-09 23:54 UTC (permalink / raw)
To: Anthony PERARD; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
Thursday, January 9, 2014, 3:56:24 PM, you wrote:
> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>> > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>> > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>> > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>> > > > [...]
>> > > > > > Those Xen report something like:
>> > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
>> > > > > > 131328
>> > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>> > > > > > memflags=0 (62 of 64)
>> > > > > >
>> > > > > > ?
>> > > > > >
>> > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
>> > > > > > QEMU :) )
>> > > > > >
>> >
>> > > -bash-4.1# lspci -s 01:00.0 -v
>> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>> > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>> > > Flags: fast devsel, IRQ 16
>> > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>> > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>> > > I/O ports at e020 [disabled] [size=32]
>> > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>> > > Expansion ROM at fb400000 [disabled] [size=4M]
>> >
>> > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>> > allocate memory for it. Will have maybe have to find another way.
>> > qemu-trad those not seems to allocate memory, but I haven't been very
>> > far in trying to check that.
>>
>> And indeed that is the case. The "Fix" below fixes it.
>>
>>
>> Based on that and this guest config:
>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>> memory = 2048
>> boot="d"
>> maxvcpus=32
>> vcpus=1
>> serial='pty'
>> vnclisten="0.0.0.0"
>> name="latest"
>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>> pci = ["01:00.0"]
>>
>> I can boot the guest.
> And can you access the ROM from the guest ?
> Also, I have another patch, it will initialize the PCI ROM BAR like any
> other BAR. In this case, if qemu is envolved in the access to ROM, it
> will print an error, like it the case for other BAR.
> I tried to test it, but it was with an embedded VGA card. When I dump
> the ROM, I got the same one as the emulated card instead of the ROM from
> the device.
Ah this is what i reported earlier ..
If you would like it more funky .. use Dario's patches to be able to use vga="none" and use:
vga="none"
nographic=1
xen_platform_pci=1
And when you dump the rombar of the passedthrough vga card, you will end up with the iPXE rom of the emulated NIC.
So it is pointing at the first / last / a random rom ... but at least it doesn't seem directly tied to pointing at
another VGA rom (which was my first assumption some time ago).
When i go one step further .. by also disabling the xen platform pci device .. it doesn't boot any more.
Strange thing is that all the addresses in debug messages (host kernel .. hvmloader seabios qemu and guest kernel) for the rom bar seem to correspond when the translation is taken into account ..
so nothing obvious there ...
--
Sander
> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> index 6dd7a68..2bbdb6d 100644
> --- a/hw/xen/xen_pt.c
> +++ b/hw/xen/xen_pt.c
> @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>
> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>
> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> - "xen-pci-pt-rom", d->rom.size);
> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> + "xen-pci-pt-rom", d->rom.size);
> pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> &s->rom);
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 23:54 ` Sander Eikelenboom
@ 2014-01-10 0:17 ` Sander Eikelenboom
2014-01-10 0:22 ` Sander Eikelenboom
0 siblings, 1 reply; 39+ messages in thread
From: Sander Eikelenboom @ 2014-01-10 0:17 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Anthony PERARD, xen-devel, stefano.stabellini, donald.d.dugger
Friday, January 10, 2014, 12:54:43 AM, you wrote:
> Thursday, January 9, 2014, 3:56:24 PM, you wrote:
>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>>> > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>>> > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>>> > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>>> > > > [...]
>>> > > > > > Those Xen report something like:
>>> > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
>>> > > > > > 131328
>>> > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>>> > > > > > memflags=0 (62 of 64)
>>> > > > > >
>>> > > > > > ?
>>> > > > > >
>>> > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
>>> > > > > > QEMU :) )
>>> > > > > >
>>> >
>>> > > -bash-4.1# lspci -s 01:00.0 -v
>>> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>>> > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>>> > > Flags: fast devsel, IRQ 16
>>> > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>>> > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>>> > > I/O ports at e020 [disabled] [size=32]
>>> > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>>> > > Expansion ROM at fb400000 [disabled] [size=4M]
>>> >
>>> > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>>> > allocate memory for it. Will have maybe have to find another way.
>>> > qemu-trad those not seems to allocate memory, but I haven't been very
>>> > far in trying to check that.
>>>
>>> And indeed that is the case. The "Fix" below fixes it.
>>>
>>>
>>> Based on that and this guest config:
>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>>> memory = 2048
>>> boot="d"
>>> maxvcpus=32
>>> vcpus=1
>>> serial='pty'
>>> vnclisten="0.0.0.0"
>>> name="latest"
>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>>> pci = ["01:00.0"]
>>>
>>> I can boot the guest.
>> And can you access the ROM from the guest ?
>> Also, I have another patch, it will initialize the PCI ROM BAR like any
>> other BAR. In this case, if qemu is envolved in the access to ROM, it
>> will print an error, like it the case for other BAR.
>> I tried to test it, but it was with an embedded VGA card. When I dump
>> the ROM, I got the same one as the emulated card instead of the ROM from
>> the device.
> Ah this is what i reported earlier ..
> If you would like it more funky .. use Dario's patches to be able to use vga="none" and use:
> vga="none"
> nographic=1
> xen_platform_pci=1
> And when you dump the rombar of the passedthrough vga card, you will end up with the iPXE rom of the emulated NIC.
> So it is pointing at the first / last / a random rom ... but at least it doesn't seem directly tied to pointing at
> another VGA rom (which was my first assumption some time ago).
> When i go one step further .. by also disabling the xen platform pci device .. it doesn't boot any more.
> Strange thing is that all the addresses in debug messages (host kernel .. hvmloader seabios qemu and guest kernel) for the rom bar seem to correspond when the translation is taken into account ..
> so nothing obvious there ...
Perhaps i have to correct myself here ... with:
vga="none"
nographic=1
xen_platform_pci=1
There seems to be a discrepancy between what the guest kernel reports at boot and the guest lspci output:
[ 0.000000] e820: [mem 0x40000000-0xfbffffff] available for PCI devices
<snip>
[ 1.453530] PCI host bridge to bus 0000:00
[ 1.460018] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.466690] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7]
[ 1.473355] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff]
[ 1.480024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[ 1.490023] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xfbffffff]
[ 1.500013] pci_bus 0000:00: scanning bus
[ 1.500504] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[ 1.500600] pci 0000:00:00.0: calling quirk_mmio_always_on+0x0/0x10
[ 1.508515] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[ 1.515216] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[ 1.531714] pci 0000:00:01.1: reg 0x20: [io 0xc240-0xc24f]
[ 1.540859] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[ 1.540859] pci 0000:00:01.3: calling acpi_pm_check_blacklist+0x0/0x40
[ 1.545776] pci 0000:00:01.3: calling quirk_piix4_acpi+0x0/0x140
[ 1.545883] pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4 ACPI
[ 1.546916] pci 0000:00:01.3: quirk: [io 0xb100-0xb10f] claimed by PIIX4 SMB
[ 1.551173] pci 0000:00:01.3: calling pci_fixup_piix4_acpi+0x0/0x10
[ 1.554237] pci 0000:00:02.0: [5853:0001] type 00 class 0xff8000
[ 1.556666] pci 0000:00:02.0: reg 0x10: [io 0xc000-0xc0ff]
[ 1.566373] pci 0000:00:02.0: reg 0x14: [mem 0xf0000000-0xf0ffffff pref]
[ 1.596780] pci 0000:00:04.0: [1002:6759] type 00 class 0x030000
[ 1.656719] pci 0000:00:04.0: reg 0x10: [mem 0xe0000000-0xefffffff 64bit pref]
[ 1.693355] pci 0000:00:04.0: reg 0x18: [mem 0xf1060000-0xf107ffff 64bit]
[ 1.743361] pci 0000:00:04.0: reg 0x20: [io 0xc100-0xc1ff]
[ 1.826692] pci 0000:00:04.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[ 1.828722] pci 0000:00:04.0: supports D1 D2
00:04.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Turks [Radeon HD 6570] (prog-if 00 [VGA controller])
Subsystem: PC Partner Limited Device e193
Physical Slot: 4
Flags: fast devsel, IRQ 32
Memory at e0000000 (64-bit, prefetchable) [size=256M]
Memory at f1060000 (64-bit, non-prefetchable) [size=128K]
I/O ports at c100 [size=256]
[virtual] Expansion ROM at f1000000 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] #1002
At least these 2 don't seem to match up:
[ 1.826692] pci 0000:00:04.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
[virtual] Expansion ROM at f1000000 [disabled] [size=128K]
> --
> Sander
>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>> index 6dd7a68..2bbdb6d 100644
>> --- a/hw/xen/xen_pt.c
>> +++ b/hw/xen/xen_pt.c
>> @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>>
>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>>
>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>> - "xen-pci-pt-rom", d->rom.size);
>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>> + "xen-pci-pt-rom", d->rom.size);
>> pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
>> &s->rom);
>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 0:17 ` Sander Eikelenboom
@ 2014-01-10 0:22 ` Sander Eikelenboom
0 siblings, 0 replies; 39+ messages in thread
From: Sander Eikelenboom @ 2014-01-10 0:22 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Anthony PERARD, xen-devel, stefano.stabellini, donald.d.dugger
Friday, January 10, 2014, 1:17:08 AM, you wrote:
> Friday, January 10, 2014, 12:54:43 AM, you wrote:
>> Thursday, January 9, 2014, 3:56:24 PM, you wrote:
>>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>>>> > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>>>> > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>>>> > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>>>> > > > [...]
>>>> > > > > > Those Xen report something like:
>>>> > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
>>>> > > > > > 131328
>>>> > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>>>> > > > > > memflags=0 (62 of 64)
>>>> > > > > >
>>>> > > > > > ?
>>>> > > > > >
>>>> > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
>>>> > > > > > QEMU :) )
>>>> > > > > >
>>>> >
>>>> > > -bash-4.1# lspci -s 01:00.0 -v
>>>> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>>>> > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>>>> > > Flags: fast devsel, IRQ 16
>>>> > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>>>> > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>>>> > > I/O ports at e020 [disabled] [size=32]
>>>> > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>>>> > > Expansion ROM at fb400000 [disabled] [size=4M]
>>>> >
>>>> > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>>>> > allocate memory for it. Will have maybe have to find another way.
>>>> > qemu-trad those not seems to allocate memory, but I haven't been very
>>>> > far in trying to check that.
>>>>
>>>> And indeed that is the case. The "Fix" below fixes it.
>>>>
>>>>
>>>> Based on that and this guest config:
>>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>>>> memory = 2048
>>>> boot="d"
>>>> maxvcpus=32
>>>> vcpus=1
>>>> serial='pty'
>>>> vnclisten="0.0.0.0"
>>>> name="latest"
>>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>>>> pci = ["01:00.0"]
>>>>
>>>> I can boot the guest.
>>> And can you access the ROM from the guest ?
>>> Also, I have another patch, it will initialize the PCI ROM BAR like any
>>> other BAR. In this case, if qemu is envolved in the access to ROM, it
>>> will print an error, like it the case for other BAR.
>>> I tried to test it, but it was with an embedded VGA card. When I dump
>>> the ROM, I got the same one as the emulated card instead of the ROM from
>>> the device.
>> Ah this is what i reported earlier ..
>> If you would like it more funky .. use Dario's patches to be able to use vga="none" and use:
>> vga="none"
>> nographic=1
>> xen_platform_pci=1
>> And when you dump the rombar of the passedthrough vga card, you will end up with the iPXE rom of the emulated NIC.
>> So it is pointing at the first / last / a random rom ... but at least it doesn't seem directly tied to pointing at
>> another VGA rom (which was my first assumption some time ago).
>> When i go one step further .. by also disabling the xen platform pci device .. it doesn't boot any more.
>> Strange thing is that all the addresses in debug messages (host kernel .. hvmloader seabios qemu and guest kernel) for the rom bar seem to correspond when the translation is taken into account ..
>> so nothing obvious there ...
> Perhaps i have to correct myself here ... with:
> vga="none"
> nographic=1
> xen_platform_pci=1
> There seems to be a discrepancy between what the guest kernel reports at boot and the guest lspci output:
> [ 0.000000] e820: [mem 0x40000000-0xfbffffff] available for PCI devices
> <snip>
> [ 1.453530] PCI host bridge to bus 0000:00
> [ 1.460018] pci_bus 0000:00: root bus resource [bus 00-ff]
> [ 1.466690] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7]
> [ 1.473355] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff]
> [ 1.480024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
> [ 1.490023] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xfbffffff]
> [ 1.500013] pci_bus 0000:00: scanning bus
> [ 1.500504] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
> [ 1.500600] pci 0000:00:00.0: calling quirk_mmio_always_on+0x0/0x10
> [ 1.508515] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
> [ 1.515216] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
> [ 1.531714] pci 0000:00:01.1: reg 0x20: [io 0xc240-0xc24f]
> [ 1.540859] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
> [ 1.540859] pci 0000:00:01.3: calling acpi_pm_check_blacklist+0x0/0x40
> [ 1.545776] pci 0000:00:01.3: calling quirk_piix4_acpi+0x0/0x140
> [ 1.545883] pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4 ACPI
> [ 1.546916] pci 0000:00:01.3: quirk: [io 0xb100-0xb10f] claimed by PIIX4 SMB
> [ 1.551173] pci 0000:00:01.3: calling pci_fixup_piix4_acpi+0x0/0x10
> [ 1.554237] pci 0000:00:02.0: [5853:0001] type 00 class 0xff8000
> [ 1.556666] pci 0000:00:02.0: reg 0x10: [io 0xc000-0xc0ff]
> [ 1.566373] pci 0000:00:02.0: reg 0x14: [mem 0xf0000000-0xf0ffffff pref]
> [ 1.596780] pci 0000:00:04.0: [1002:6759] type 00 class 0x030000
> [ 1.656719] pci 0000:00:04.0: reg 0x10: [mem 0xe0000000-0xefffffff 64bit pref]
> [ 1.693355] pci 0000:00:04.0: reg 0x18: [mem 0xf1060000-0xf107ffff 64bit]
> [ 1.743361] pci 0000:00:04.0: reg 0x20: [io 0xc100-0xc1ff]
> [ 1.826692] pci 0000:00:04.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> [ 1.828722] pci 0000:00:04.0: supports D1 D2
> 00:04.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Turks [Radeon HD 6570] (prog-if 00 [VGA controller])
> Subsystem: PC Partner Limited Device e193
> Physical Slot: 4
> Flags: fast devsel, IRQ 32
> Memory at e0000000 (64-bit, prefetchable) [size=256M]
> Memory at f1060000 (64-bit, non-prefetchable) [size=128K]
> I/O ports at c100 [size=256]
> [virtual] Expansion ROM at f1000000 [disabled] [size=128K]
> Capabilities: [50] Power Management version 3
> Capabilities: [58] Express Legacy Endpoint, MSI 00
> Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
> Capabilities: [100] #1002
> At least these 2 don't seem to match up:
> [ 1.826692] pci 0000:00:04.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> [virtual] Expansion ROM at f1000000 [disabled] [size=128K]
And from xl dmesg:
(d23) [2014-01-10 00:07:47] pci dev 03:0 bar 30 size 000040000: 0f1000000
which is .. probably the NIC ... but which is mysteriously never shown in lspci (thought i had reported that earlier as well):
lspci -v
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
Subsystem: Red Hat, Inc Qemu virtual machine
Flags: bus master, fast devsel, latency 0
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
Subsystem: Red Hat, Inc Qemu virtual machine
Physical Slot: 1
Flags: bus master, medium devsel, latency 0
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] (prog-if 80 [Master])
Subsystem: Red Hat, Inc Qemu virtual machine
Physical Slot: 1
Flags: bus master, medium devsel, latency 0
[virtual] Memory at 000001f0 (32-bit, non-prefetchable) [size=8]
[virtual] Memory at 000003f0 (type 3, non-prefetchable) [size=1]
[virtual] Memory at 00000170 (32-bit, non-prefetchable) [size=8]
[virtual] Memory at 00000370 (type 3, non-prefetchable) [size=1]
I/O ports at c240 [size=16]
Kernel driver in use: PIIX_IDE
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
Subsystem: Red Hat, Inc Qemu virtual machine
Physical Slot: 1
Flags: bus master, medium devsel, latency 0, IRQ 9
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
Subsystem: XenSource, Inc. Xen Platform Device
Physical Slot: 2
Flags: bus master, fast devsel, latency 0, IRQ 24
I/O ports at c000 [size=256]
Memory at f0000000 (32-bit, prefetchable) [size=16M]
Kernel driver in use: xen-platform-pci
00:04.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Turks [Radeon HD 6570] (prog-if 00 [VGA controller])
Subsystem: PC Partner Limited Device e193
Physical Slot: 4
Flags: fast devsel, IRQ 32
Memory at e0000000 (64-bit, prefetchable) [size=256M]
Memory at f1060000 (64-bit, non-prefetchable) [size=128K]
I/O ports at c100 [size=256]
[virtual] Expansion ROM at f1000000 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] #1002
which is defined in the guest config as:
vif = [ 'bridge=xen_bridge,ip=192.168.1.44,mac=00:16:3A:C6:76:65, model=e1000' ]
>> --
>> Sander
>>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>>> index 6dd7a68..2bbdb6d 100644
>>> --- a/hw/xen/xen_pt.c
>>> +++ b/hw/xen/xen_pt.c
>>> @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>>>
>>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>>>
>>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>>> - "xen-pci-pt-rom", d->rom.size);
>>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>>> + "xen-pci-pt-rom", d->rom.size);
>>> pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
>>> &s->rom);
>>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 14:56 ` Anthony PERARD
2014-01-09 15:36 ` Pasi Kärkkäinen
2014-01-09 23:54 ` Sander Eikelenboom
@ 2014-01-10 3:28 ` Konrad Rzeszutek Wilk
2014-01-10 15:19 ` Anthony PERARD
2014-01-26 0:05 ` Zhang, Yang Z
2014-01-26 2:15 ` Zhang, Yang Z
4 siblings, 1 reply; 39+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-01-10 3:28 UTC (permalink / raw)
To: Anthony PERARD; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
On Thu, Jan 09, 2014 at 02:56:24PM +0000, Anthony PERARD wrote:
> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> > > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
> > > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> > > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> > > > > [...]
> > > > > > > Those Xen report something like:
> > > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
> > > > > > > 131328
> > > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
> > > > > > > memflags=0 (62 of 64)
> > > > > > >
> > > > > > > ?
> > > > > > >
> > > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
> > > > > > > QEMU :) )
> > > > > > >
> > >
> > > > -bash-4.1# lspci -s 01:00.0 -v
> > > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> > > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> > > > Flags: fast devsel, IRQ 16
> > > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
> > > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> > > > I/O ports at e020 [disabled] [size=32]
> > > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
> > > > Expansion ROM at fb400000 [disabled] [size=4M]
> > >
> > > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> > > allocate memory for it. Will have maybe have to find another way.
> > > qemu-trad those not seems to allocate memory, but I haven't been very
> > > far in trying to check that.
> >
> > And indeed that is the case. The "Fix" below fixes it.
> >
> >
> > Based on that and this guest config:
> > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> > memory = 2048
> > boot="d"
> > maxvcpus=32
> > vcpus=1
> > serial='pty'
> > vnclisten="0.0.0.0"
> > name="latest"
> > vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
> > pci = ["01:00.0"]
> >
> > I can boot the guest.
>
> And can you access the ROM from the guest ?
I hadn't tried it. This is with a NIC and I just wanted to see if it
could do PCI passthrough without using the Option ROM.
>
>
> Also, I have another patch, it will initialize the PCI ROM BAR like any
> other BAR. In this case, if qemu is envolved in the access to ROM, it
> will print an error, like it the case for other BAR.
>
> I tried to test it, but it was with an embedded VGA card. When I dump
> the ROM, I got the same one as the emulated card instead of the ROM from
> the device.
Oddly enough for me with your patch the NIC's BIOS was invoked and
it tried to PXE boot:
(d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
(d1) [2014-01-10 03:20:47] Booting from DVD/CD...
(d1) [2014-01-10 03:20:47] Booting from 0000:7c00
..
and I did see the PXE boot menu in my guest - so even
better!
I have not yet done the GPU - this issue was preventing me from using
qemu-xen as it would always blow up before SeaBIOS was in the picture.
If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com>' please do.
Thank you!
>
>
> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> index 6dd7a68..2bbdb6d 100644
> --- a/hw/xen/xen_pt.c
> +++ b/hw/xen/xen_pt.c
> @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>
> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>
> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> - "xen-pci-pt-rom", d->rom.size);
> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> + "xen-pci-pt-rom", d->rom.size);
> pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> &s->rom);
>
>
> --
> Anthony PERARD
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 3:28 ` Konrad Rzeszutek Wilk
@ 2014-01-10 15:19 ` Anthony PERARD
2014-01-10 16:05 ` Sander Eikelenboom
0 siblings, 1 reply; 39+ messages in thread
From: Anthony PERARD @ 2014-01-10 15:19 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
On Thu, Jan 09, 2014 at 10:28:47PM -0500, Konrad Rzeszutek Wilk wrote:
> On Thu, Jan 09, 2014 at 02:56:24PM +0000, Anthony PERARD wrote:
> > On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> > > On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> > > > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
> > > > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> > > > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> > > > > > [...]
> > > > > > > > Those Xen report something like:
> > > > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
> > > > > > > > 131328
> > > > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
> > > > > > > > memflags=0 (62 of 64)
> > > > > > > >
> > > > > > > > ?
> > > > > > > >
> > > > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
> > > > > > > > QEMU :) )
> > > > > > > >
> > > >
> > > > > -bash-4.1# lspci -s 01:00.0 -v
> > > > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> > > > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> > > > > Flags: fast devsel, IRQ 16
> > > > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
> > > > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> > > > > I/O ports at e020 [disabled] [size=32]
> > > > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
> > > > > Expansion ROM at fb400000 [disabled] [size=4M]
> > > >
> > > > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> > > > allocate memory for it. Will have maybe have to find another way.
> > > > qemu-trad those not seems to allocate memory, but I haven't been very
> > > > far in trying to check that.
> > >
> > > And indeed that is the case. The "Fix" below fixes it.
> > >
> > >
> > > Based on that and this guest config:
> > > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> > > memory = 2048
> > > boot="d"
> > > maxvcpus=32
> > > vcpus=1
> > > serial='pty'
> > > vnclisten="0.0.0.0"
> > > name="latest"
> > > vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
> > > pci = ["01:00.0"]
> > >
> > > I can boot the guest.
> >
> > And can you access the ROM from the guest ?
>
> I hadn't tried it. This is with a NIC and I just wanted to see if it
> could do PCI passthrough without using the Option ROM.
> >
> >
> > Also, I have another patch, it will initialize the PCI ROM BAR like any
> > other BAR. In this case, if qemu is envolved in the access to ROM, it
> > will print an error, like it the case for other BAR.
> >
> > I tried to test it, but it was with an embedded VGA card. When I dump
> > the ROM, I got the same one as the emulated card instead of the ROM from
> > the device.
>
> Oddly enough for me with your patch the NIC's BIOS was invoked and
> it tried to PXE boot:
>
> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
>
> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
> ..
> and I did see the PXE boot menu in my guest - so even
> better!
Perfect, look like it is the fix for PCI passthrough.
> I have not yet done the GPU - this issue was preventing me from using
> qemu-xen as it would always blow up before SeaBIOS was in the picture.
>
> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com>' please do.
Will do.
> Thank you!
> >
> >
> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> > index 6dd7a68..2bbdb6d 100644
> > --- a/hw/xen/xen_pt.c
> > +++ b/hw/xen/xen_pt.c
> > @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
> >
> > s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> >
> > - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> > - "xen-pci-pt-rom", d->rom.size);
> > + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> > + "xen-pci-pt-rom", d->rom.size);
> > pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> > &s->rom);
> >
> >
> > --
> > Anthony PERARD
--
Anthony PERARD
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 15:19 ` Anthony PERARD
@ 2014-01-10 16:05 ` Sander Eikelenboom
2014-01-10 16:25 ` Anthony PERARD
2014-01-10 16:35 ` Konrad Rzeszutek Wilk
0 siblings, 2 replies; 39+ messages in thread
From: Sander Eikelenboom @ 2014-01-10 16:05 UTC (permalink / raw)
To: Anthony PERARD; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
Friday, January 10, 2014, 4:19:15 PM, you wrote:
> On Thu, Jan 09, 2014 at 10:28:47PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Thu, Jan 09, 2014 at 02:56:24PM +0000, Anthony PERARD wrote:
>> > On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>> > > On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>> > > > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>> > > > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>> > > > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>> > > > > > [...]
>> > > > > > > > Those Xen report something like:
>> > > > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
>> > > > > > > > 131328
>> > > > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>> > > > > > > > memflags=0 (62 of 64)
>> > > > > > > >
>> > > > > > > > ?
>> > > > > > > >
>> > > > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in
>> > > > > > > > QEMU :) )
>> > > > > > > >
>> > > >
>> > > > > -bash-4.1# lspci -s 01:00.0 -v
>> > > > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>> > > > > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>> > > > > Flags: fast devsel, IRQ 16
>> > > > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>> > > > > Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>> > > > > I/O ports at e020 [disabled] [size=32]
>> > > > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>> > > > > Expansion ROM at fb400000 [disabled] [size=4M]
>> > > >
>> > > > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>> > > > allocate memory for it. Will have maybe have to find another way.
>> > > > qemu-trad those not seems to allocate memory, but I haven't been very
>> > > > far in trying to check that.
>> > >
>> > > And indeed that is the case. The "Fix" below fixes it.
>> > >
>> > >
>> > > Based on that and this guest config:
>> > > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>> > > memory = 2048
>> > > boot="d"
>> > > maxvcpus=32
>> > > vcpus=1
>> > > serial='pty'
>> > > vnclisten="0.0.0.0"
>> > > name="latest"
>> > > vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>> > > pci = ["01:00.0"]
>> > >
>> > > I can boot the guest.
>> >
>> > And can you access the ROM from the guest ?
>>
>> I hadn't tried it. This is with a NIC and I just wanted to see if it
>> could do PCI passthrough without using the Option ROM.
>> >
>> >
>> > Also, I have another patch, it will initialize the PCI ROM BAR like any
>> > other BAR. In this case, if qemu is envolved in the access to ROM, it
>> > will print an error, like it the case for other BAR.
>> >
>> > I tried to test it, but it was with an embedded VGA card. When I dump
>> > the ROM, I got the same one as the emulated card instead of the ROM from
>> > the device.
>>
>> Oddly enough for me with your patch the NIC's BIOS was invoked and
>> it tried to PXE boot:
>>
>> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
>>
>> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
>> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
>> ..
>> and I did see the PXE boot menu in my guest - so even
>> better!
> Perfect, look like it is the fix for PCI passthrough.
Hi Konrad,
Are you sure it's the rom of the NIC, and not the iPXE rom from the emulated device that
gets run ?
With this patch and VGA devices it still points to another rom for me.
(it looks like it is pointing to the rom of the device with a BDF just one lower than the passed through one)
Do you by any chance know if there is a difference in how lspci and the linux kernel scan / list pci devices ?
(for example one by reading acpi tables .. the other one by real probing .. ?)
--
Sander
>> I have not yet done the GPU - this issue was preventing me from using
>> qemu-xen as it would always blow up before SeaBIOS was in the picture.
>>
>> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
>> <konrad.wilk@oracle.com>' please do.
> Will do.
>> Thank you!
>> >
>> >
>> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>> > index 6dd7a68..2bbdb6d 100644
>> > --- a/hw/xen/xen_pt.c
>> > +++ b/hw/xen/xen_pt.c
>> > @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>> >
>> > s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>> >
>> > - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>> > - "xen-pci-pt-rom", d->rom.size);
>> > + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>> > + "xen-pci-pt-rom", d->rom.size);
>> > pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
>> > &s->rom);
>> >
>> >
>> > --
>> > Anthony PERARD
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 16:05 ` Sander Eikelenboom
@ 2014-01-10 16:25 ` Anthony PERARD
2014-01-10 16:38 ` Sander Eikelenboom
2014-01-10 16:35 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 39+ messages in thread
From: Anthony PERARD @ 2014-01-10 16:25 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
On Fri, Jan 10, 2014 at 05:05:44PM +0100, Sander Eikelenboom wrote:
> With this patch and VGA devices it still points to another rom for me.
> (it looks like it is pointing to the rom of the device with a BDF just one lower than the passed through one)
I do think VGA devices are a particular case. hvmloader or SeaBIOS might
"replace" the ROM of the card.
--
Anthony PERARD
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 16:25 ` Anthony PERARD
@ 2014-01-10 16:38 ` Sander Eikelenboom
0 siblings, 0 replies; 39+ messages in thread
From: Sander Eikelenboom @ 2014-01-10 16:38 UTC (permalink / raw)
To: Anthony PERARD; +Cc: xen-devel, stefano.stabellini, donald.d.dugger
Friday, January 10, 2014, 5:25:35 PM, you wrote:
> On Fri, Jan 10, 2014 at 05:05:44PM +0100, Sander Eikelenboom wrote:
>> With this patch and VGA devices it still points to another rom for me.
>> (it looks like it is pointing to the rom of the device with a BDF just one lower than the passed through one)
> I do think VGA devices are a particular case. hvmloader or SeaBIOS might
> "replace" the ROM of the card.
Any way / patch to find out ?
It would be quite silly if it replaces the vga rom with a nic rom.
If that only happens for vga cards .. it would need to check for device class,
would seem very strange to replace it with another rom and not check for class there.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 16:05 ` Sander Eikelenboom
2014-01-10 16:25 ` Anthony PERARD
@ 2014-01-10 16:35 ` Konrad Rzeszutek Wilk
2014-01-10 17:38 ` Sander Eikelenboom
1 sibling, 1 reply; 39+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-01-10 16:35 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Anthony PERARD, xen-devel, stefano.stabellini, donald.d.dugger
> >> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
> >>
> >> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
> >> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
> >> ..
> >> and I did see the PXE boot menu in my guest - so even
> >> better!
>
> > Perfect, look like it is the fix for PCI passthrough.
>
> Hi Konrad,
>
> Are you sure it's the rom of the NIC, and not the iPXE rom from the emulated device that
> gets run ?
Yes. I double checked that the MAC address that was given an DHCP
address was indeed for the physical hardware. And it was.
>
> With this patch and VGA devices it still points to another rom for me.
> (it looks like it is pointing to the rom of the device with a BDF just one lower than the passed through one)
That all sounds to me like a bug in QEMU which constructs
the 'world'. Then 'hvmloader' and the kernel ingest this to
create their view of what the PCI configuration/slots/etc should look like.
They use 'inb' and 'outb' instructions on the 0xcf8/0xcfc port.
Perhaps the 'vga=none' is having an hard time dealing with 'no-VGA-but-wait-there
is-and-PT-VGA'?
Just to make sure I am not forgetting a crucial fact - if you don't
have vga=none, does the lspci output look sane?
>
> Do you by any chance know if there is a difference in how lspci and the linux kernel scan / list pci devices ?
> (for example one by reading acpi tables .. the other one by real probing .. ?)
The kernel and hvmloader all use the 'inb' and 'outb' to figure out
what the PCI space looks like.
'lspci' uses /sysfs. Thought if you use '-xxxx' I think it also does
'inb' and 'outb'.
So it all seems to point to QEMU.
>
> --
> Sander
>
>
> >> I have not yet done the GPU - this issue was preventing me from using
> >> qemu-xen as it would always blow up before SeaBIOS was in the picture.
> >>
> >> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
> >> <konrad.wilk@oracle.com>' please do.
>
> > Will do.
>
> >> Thank you!
> >> >
> >> >
> >> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> >> > index 6dd7a68..2bbdb6d 100644
> >> > --- a/hw/xen/xen_pt.c
> >> > +++ b/hw/xen/xen_pt.c
> >> > @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
> >> >
> >> > s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> >> >
> >> > - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> >> > - "xen-pci-pt-rom", d->rom.size);
> >> > + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> >> > + "xen-pci-pt-rom", d->rom.size);
> >> > pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> >> > &s->rom);
> >> >
> >> >
> >> > --
> >> > Anthony PERARD
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 16:35 ` Konrad Rzeszutek Wilk
@ 2014-01-10 17:38 ` Sander Eikelenboom
2014-01-10 18:01 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 39+ messages in thread
From: Sander Eikelenboom @ 2014-01-10 17:38 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Anthony PERARD, xen-devel, stefano.stabellini, donald.d.dugger
Friday, January 10, 2014, 5:35:44 PM, you wrote:
>> >> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
>> >>
>> >> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
>> >> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
>> >> ..
>> >> and I did see the PXE boot menu in my guest - so even
>> >> better!
>>
>> > Perfect, look like it is the fix for PCI passthrough.
>>
>> Hi Konrad,
>>
>> Are you sure it's the rom of the NIC, and not the iPXE rom from the emulated device that
>> gets run ?
> Yes. I double checked that the MAC address that was given an DHCP
> address was indeed for the physical hardware. And it was.
OK
>>
>> With this patch and VGA devices it still points to another rom for me.
>> (it looks like it is pointing to the rom of the device with a BDF just one lower than the passed through one)
> That all sounds to me like a bug in QEMU which constructs
> the 'world'. Then 'hvmloader' and the kernel ingest this to
> create their view of what the PCI configuration/slots/etc should look like.
> They use 'inb' and 'outb' instructions on the 0xcf8/0xcfc port.
> Perhaps the 'vga=none' is having an hard time dealing with 'no-VGA-but-wait-there
> is-and-PT-VGA'?
> Just to make sure I am not forgetting a crucial fact - if you don't
> have vga=none, does the lspci output look sane?
It does .. except when using an PV NIC, that one doesn't show in lspci, but it does occupy a slot (the numbers are not
consecutive anymore, one is "hidden"), but the PV nic itself is working ok.
This doesn't seem to happen for disk so that seems a bit strange to me ..
Apart from that it looks sane.
But I'm rebuilding everything now from scratch and will try again .. just too damned many parameters :-)
Will see if i can make a complete post again with all data, although the previous time i tried that,
it was a little bit to intimidating i guess.
>>
>> Do you by any chance know if there is a difference in how lspci and the linux kernel scan / list pci devices ?
>> (for example one by reading acpi tables .. the other one by real probing .. ?)
> The kernel and hvmloader all use the 'inb' and 'outb' to figure out
> what the PCI space looks like.
> 'lspci' uses /sysfs. Thought if you use '-xxxx' I think it also does
> 'inb' and 'outb'.
Hmm ok .. so i would expect that dumping with the:
echo 1 > rom; cat rom > rom.bin; echo 0 > rom;
sequence in /sys/bus/pci/devices/<BDF> would do everything according to the addresses that lspci gives for the rombar ..
and when i do that the resulting rom.bin differs according to the emulated devices i put it.
The devices that have a rom in the guest are always in the order (with there BDF):
NIC
Emulated VGA
Passthroughed VGA
When all enabled, i end up with the rom of the emulated VGA
When i disable that with vga="none", i end up with the rom of the NIC
When i also disable that .. by not specifying any vif and using xen_platform_pci=0 ..
the kernel complains when trying to dump that it's not a valid rom (and when forced too it's all zero's so it's
right because it can't find the start signature of a rom)
Is there an easy tool to dump the content of arbitrary mem addresses ?
(except enabling the /dev/kmem and trying to do some voodoo calculations and use "dd" ?)
Hmm when i make a core dump using "xl dump-core" would/should that also contain the content of the passed throughed rom ?
Then at least i could search that dump for the strings of the real vga rom of the passedtroughed card and see if it is there .. perhaps on some other
address as expected .. or at least know it isn't.
Because at the moment i'm not very well in progressing with ruling things out (which seems to be the only method),
i only have some symptoms that differ. Wish i had another NIC device with a rom to see if that does work.
> So it all seems to point to QEMU.
>>
>> --
>> Sander
>>
>>
>> >> I have not yet done the GPU - this issue was preventing me from using
>> >> qemu-xen as it would always blow up before SeaBIOS was in the picture.
>> >>
>> >> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
>> >> <konrad.wilk@oracle.com>' please do.
>>
>> > Will do.
>>
>> >> Thank you!
>> >> >
>> >> >
>> >> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>> >> > index 6dd7a68..2bbdb6d 100644
>> >> > --- a/hw/xen/xen_pt.c
>> >> > +++ b/hw/xen/xen_pt.c
>> >> > @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
>> >> >
>> >> > s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>> >> >
>> >> > - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>> >> > - "xen-pci-pt-rom", d->rom.size);
>> >> > + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>> >> > + "xen-pci-pt-rom", d->rom.size);
>> >> > pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
>> >> > &s->rom);
>> >> >
>> >> >
>> >> > --
>> >> > Anthony PERARD
>>
>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-10 17:38 ` Sander Eikelenboom
@ 2014-01-10 18:01 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 39+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-01-10 18:01 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Anthony PERARD, xen-devel, stefano.stabellini, donald.d.dugger
On Fri, Jan 10, 2014 at 06:38:18PM +0100, Sander Eikelenboom wrote:
>
> Friday, January 10, 2014, 5:35:44 PM, you wrote:
>
> >> >> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
> >> >>
> >> >> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
> >> >> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
> >> >> ..
> >> >> and I did see the PXE boot menu in my guest - so even
> >> >> better!
> >>
> >> > Perfect, look like it is the fix for PCI passthrough.
> >>
> >> Hi Konrad,
> >>
> >> Are you sure it's the rom of the NIC, and not the iPXE rom from the emulated device that
> >> gets run ?
>
> > Yes. I double checked that the MAC address that was given an DHCP
> > address was indeed for the physical hardware. And it was.
>
> OK
>
> >>
> >> With this patch and VGA devices it still points to another rom for me.
> >> (it looks like it is pointing to the rom of the device with a BDF just one lower than the passed through one)
>
> > That all sounds to me like a bug in QEMU which constructs
> > the 'world'. Then 'hvmloader' and the kernel ingest this to
> > create their view of what the PCI configuration/slots/etc should look like.
> > They use 'inb' and 'outb' instructions on the 0xcf8/0xcfc port.
>
> > Perhaps the 'vga=none' is having an hard time dealing with 'no-VGA-but-wait-there
> > is-and-PT-VGA'?
>
> > Just to make sure I am not forgetting a crucial fact - if you don't
> > have vga=none, does the lspci output look sane?
>
> It does .. except when using an PV NIC, that one doesn't show in lspci, but it does occupy a slot (the numbers are not
> consecutive anymore, one is "hidden"), but the PV nic itself is working ok.
Right, because your Intel emulated gets 'unplugged' (disappears)
when the Xen PV one kicks in.
> This doesn't seem to happen for disk so that seems a bit strange to me ..
That might be due to the CD-ROM option. The disk should
nonetless disappear (/dev/hda).
>
> Apart from that it looks sane.
>
> But I'm rebuilding everything now from scratch and will try again .. just too damned many parameters :-)
>
> Will see if i can make a complete post again with all data, although the previous time i tried that,
> it was a little bit to intimidating i guess.
>
> >>
> >> Do you by any chance know if there is a difference in how lspci and the linux kernel scan / list pci devices ?
> >> (for example one by reading acpi tables .. the other one by real probing .. ?)
>
> > The kernel and hvmloader all use the 'inb' and 'outb' to figure out
> > what the PCI space looks like.
>
> > 'lspci' uses /sysfs. Thought if you use '-xxxx' I think it also does
> > 'inb' and 'outb'.
>
> Hmm ok .. so i would expect that dumping with the:
> echo 1 > rom; cat rom > rom.bin; echo 0 > rom;
> sequence in /sys/bus/pci/devices/<BDF> would do everything according to the addresses that lspci gives for the rombar ..
> and when i do that the resulting rom.bin differs according to the emulated devices i put it.
>
> The devices that have a rom in the guest are always in the order (with there BDF):
> NIC
> Emulated VGA
> Passthroughed VGA
>
> When all enabled, i end up with the rom of the emulated VGA
> When i disable that with vga="none", i end up with the rom of the NIC
> When i also disable that .. by not specifying any vif and using xen_platform_pci=0 ..
> the kernel complains when trying to dump that it's not a valid rom (and when forced too it's all zero's so it's
> right because it can't find the start signature of a rom)
>
Ok,so the passthrough ROM is definitly not showing up. Which is
odd, b/c it does show up for the physical NIC.
If you do the 'echo 1' .. rune in dom0, can you extract your
VGA (Radeon) BIOS? Perhaps the ROM extraction part for video
cards is different?
> Is there an easy tool to dump the content of arbitrary mem addresses ?
> (except enabling the /dev/kmem and trying to do some voodoo calculations and use "dd" ?)
That is the easiest.
>
> Hmm when i make a core dump using "xl dump-core" would/should that also contain the content of the passed throughed rom ?
I think not. As the ROM is not RAM it should not include that.
> Then at least i could search that dump for the strings of the real vga rom of the passedtroughed card and see if it is there .. perhaps on some other
> address as expected .. or at least know it isn't.
>
> Because at the moment i'm not very well in progressing with ruling things out (which seems to be the only method),
> i only have some symptoms that differ. Wish i had another NIC device with a rom to see if that does work.
>
>
> > So it all seems to point to QEMU.
> >>
> >> --
> >> Sander
> >>
> >>
> >> >> I have not yet done the GPU - this issue was preventing me from using
> >> >> qemu-xen as it would always blow up before SeaBIOS was in the picture.
> >> >>
> >> >> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
> >> >> <konrad.wilk@oracle.com>' please do.
> >>
> >> > Will do.
> >>
> >> >> Thank you!
> >> >> >
> >> >> >
> >> >> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> >> >> > index 6dd7a68..2bbdb6d 100644
> >> >> > --- a/hw/xen/xen_pt.c
> >> >> > +++ b/hw/xen/xen_pt.c
> >> >> > @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState *s)
> >> >> >
> >> >> > s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> >> >> >
> >> >> > - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> >> >> > - "xen-pci-pt-rom", d->rom.size);
> >> >> > + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> >> >> > + "xen-pci-pt-rom", d->rom.size);
> >> >> > pci_register_bar(&s->dev, PCI_ROM_SLOT, PCI_BASE_ADDRESS_MEM_PREFETCH,
> >> >> > &s->rom);
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Anthony PERARD
> >>
> >>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 14:56 ` Anthony PERARD
` (2 preceding siblings ...)
2014-01-10 3:28 ` Konrad Rzeszutek Wilk
@ 2014-01-26 0:05 ` Zhang, Yang Z
2014-01-27 12:12 ` Stefano Stabellini
2014-01-26 2:15 ` Zhang, Yang Z
4 siblings, 1 reply; 39+ messages in thread
From: Zhang, Yang Z @ 2014-01-26 0:05 UTC (permalink / raw)
To: Anthony PERARD, Konrad Rzeszutek Wilk
Cc: xen-devel, stefano.stabellini, Dugger, Donald D
Anthony PERARD wrote on 2014-01-09:
> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>>> On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>>>> On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>>>>> On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>>>>> [...]
>>>>>>> Those Xen report something like:
>>>>>>> (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46:
>>>>>>> 131329 >
>>>>>>> 131328
>>>>>>> (XEN) memory.c:132:d0 Could not allocate order=0 extent:
>>>>>>> id=46
>>>>>>> memflags=0 (62 of 64)
>>>>>>>
>>>>>>> ?
>>>>>>>
>>>>>>> (I tryied to reproduce the issue by simply add many
>>>>>>> emulated
>>>>>>> e1000 in QEMU :) )
>>>>>>>
>>>
>>>> -bash-4.1# lspci -s 01:00.0 -v
>>>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
>>>> Network
> Connection (rev 01)
>>>> Subsystem: Intel Corporation Gigabit ET Dual Port Server
>>>> Adapter Flags: fast devsel, IRQ 16 Memory at fbc20000
>>>> (32-bit, non-prefetchable) [disabled] [size=128K] Memory at
>>>> fb800000 (32-bit, non-prefetchable) [disabled] [size=4M] I/O
>>>> ports at e020 [disabled] [size=32] Memory at fbc44000
>>>> (32-bit, non-prefetchable) [disabled] [size=16K] Expansion
>>>> ROM at fb400000 [disabled] [size=4M]
>>>
>>> BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>>> allocate memory for it. Will have maybe have to find another way.
>>> qemu-trad those not seems to allocate memory, but I haven't been
>>> very far in trying to check that.
>>
>> And indeed that is the case. The "Fix" below fixes it.
>>
>>
>> Based on that and this guest config:
>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>> memory = 2048
>> boot="d"
>> maxvcpus=32
>> vcpus=1
>> serial='pty'
>> vnclisten="0.0.0.0"
>> name="latest"
>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ] pci = ["01:00.0"]
>>
>> I can boot the guest.
>
> And can you access the ROM from the guest ?
>
>
> Also, I have another patch, it will initialize the PCI ROM BAR like any other BAR.
> In this case, if qemu is envolved in the access to ROM, it will print
> an error, like it the case for other BAR.
>
> I tried to test it, but it was with an embedded VGA card. When I dump
> the ROM, I got the same one as the emulated card instead of the ROM from the device.
>
>
> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index 6dd7a68..2bbdb6d
> 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -440,8 +440,8 @@
> static int xen_pt_register_regions(XenPCIPassthroughState *s)
>
> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> - "xen-pci-pt-rom", d->rom.size);
> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> + "xen-pci-pt-rom", d->rom.size);
> pci_register_bar(&s->dev, PCI_ROM_SLOT,
> PCI_BASE_ADDRESS_MEM_PREFETCH,
> &s->rom);
>
Hi, Anthony,
Does your fixing is the final solution for this issue? If yes, will you push it before Xen 4.4 release?
Best regards,
Yang
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-26 0:05 ` Zhang, Yang Z
@ 2014-01-27 12:12 ` Stefano Stabellini
2014-01-27 15:34 ` Zhang, Yang Z
0 siblings, 1 reply; 39+ messages in thread
From: Stefano Stabellini @ 2014-01-27 12:12 UTC (permalink / raw)
To: Zhang, Yang Z
Cc: Anthony PERARD, xen-devel, stefano.stabellini, Dugger, Donald D
On Sun, 26 Jan 2014, Zhang, Yang Z wrote:
> Anthony PERARD wrote on 2014-01-09:
> > On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> >> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> >>> On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
> >>>> On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> >>>>> On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> >>>>> [...]
> >>>>>>> Those Xen report something like:
> >>>>>>> (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46:
> >>>>>>> 131329 >
> >>>>>>> 131328
> >>>>>>> (XEN) memory.c:132:d0 Could not allocate order=0 extent:
> >>>>>>> id=46
> >>>>>>> memflags=0 (62 of 64)
> >>>>>>>
> >>>>>>> ?
> >>>>>>>
> >>>>>>> (I tryied to reproduce the issue by simply add many
> >>>>>>> emulated
> >>>>>>> e1000 in QEMU :) )
> >>>>>>>
> >>>
> >>>> -bash-4.1# lspci -s 01:00.0 -v
> >>>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> >>>> Network
> > Connection (rev 01)
> >>>> Subsystem: Intel Corporation Gigabit ET Dual Port Server
> >>>> Adapter Flags: fast devsel, IRQ 16 Memory at fbc20000
> >>>> (32-bit, non-prefetchable) [disabled] [size=128K] Memory at
> >>>> fb800000 (32-bit, non-prefetchable) [disabled] [size=4M] I/O
> >>>> ports at e020 [disabled] [size=32] Memory at fbc44000
> >>>> (32-bit, non-prefetchable) [disabled] [size=16K] Expansion
> >>>> ROM at fb400000 [disabled] [size=4M]
> >>>
> >>> BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> >>> allocate memory for it. Will have maybe have to find another way.
> >>> qemu-trad those not seems to allocate memory, but I haven't been
> >>> very far in trying to check that.
> >>
> >> And indeed that is the case. The "Fix" below fixes it.
> >>
> >>
> >> Based on that and this guest config:
> >> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> >> memory = 2048
> >> boot="d"
> >> maxvcpus=32
> >> vcpus=1
> >> serial='pty'
> >> vnclisten="0.0.0.0"
> >> name="latest"
> >> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ] pci = ["01:00.0"]
> >>
> >> I can boot the guest.
> >
> > And can you access the ROM from the guest ?
> >
> >
> > Also, I have another patch, it will initialize the PCI ROM BAR like any other BAR.
> > In this case, if qemu is envolved in the access to ROM, it will print
> > an error, like it the case for other BAR.
> >
> > I tried to test it, but it was with an embedded VGA card. When I dump
> > the ROM, I got the same one as the emulated card instead of the ROM from the device.
> >
> >
> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index 6dd7a68..2bbdb6d
> > 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -440,8 +440,8 @@
> > static int xen_pt_register_regions(XenPCIPassthroughState *s)
> >
> > s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> > - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> > - "xen-pci-pt-rom", d->rom.size);
> > + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> > + "xen-pci-pt-rom", d->rom.size);
> > pci_register_bar(&s->dev, PCI_ROM_SLOT,
> > PCI_BASE_ADDRESS_MEM_PREFETCH,
> > &s->rom);
> >
>
> Hi, Anthony,
>
> Does your fixing is the final solution for this issue? If yes, will you push it before Xen 4.4 release?
I included this patch in the last pull request I sent to Anthony
Liguori:
http://marc.info/?l=qemu-devel&m=138997319906095
It hasn't been pulled yet, but I would expect that it is going to be
upstream soon.
Regarding the 4.4 release, we are trying to fix a couple of other
serious bugs in the qemu-xen tree right now, but it is still conceivable
to have this fix backported to the tree in time for the release.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-27 12:12 ` Stefano Stabellini
@ 2014-01-27 15:34 ` Zhang, Yang Z
2014-01-27 15:37 ` Stefano Stabellini
2014-01-31 15:23 ` Stefano Stabellini
0 siblings, 2 replies; 39+ messages in thread
From: Zhang, Yang Z @ 2014-01-27 15:34 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Anthony PERARD, xen-devel, stefano.stabellini, Dugger, Donald D
Stefano Stabellini wrote on 2014-01-27:
> On Sun, 26 Jan 2014, Zhang, Yang Z wrote:
>> Anthony PERARD wrote on 2014-01-09:
>>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>>>>> On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk
> wrote:
>>>>>> On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>>>>>>> On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>>>>>>> [...]
>>>>>>>>> Those Xen report something like:
>>>>>>>>> (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46:
>>>>>>>>> 131329 >
>>>>>>>>> 131328
>>>>>>>>> (XEN) memory.c:132:d0 Could not allocate order=0 extent:
>>>>>>>>> id=46
>>>>>>>>> memflags=0 (62 of 64)
>>>>>>>>>
>>>>>>>>> ?
>>>>>>>>>
>>>>>>>>> (I tryied to reproduce the issue by simply add many emulated
>>>>>>>>> e1000 in QEMU :) )
>>>>>>>>>
>>>>>
>>>>>> -bash-4.1# lspci -s 01:00.0 -v
>>>>>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
>>>>>> Network
>>> Connection (rev 01)
>>>>>> Subsystem: Intel Corporation Gigabit ET Dual Port Server
>>>>>> Adapter Flags: fast devsel, IRQ 16 Memory at fbc20000
>>>>>> (32-bit, non-prefetchable) [disabled] [size=128K] Memory at
>>>>>> fb800000 (32-bit, non-prefetchable) [disabled] [size=4M] I/O
>>>>>> ports at e020 [disabled] [size=32] Memory at fbc44000
>>>>>> (32-bit, non-prefetchable) [disabled] [size=16K] Expansion
>>>>>> ROM at fb400000 [disabled] [size=4M]
>>>>>
>>>>> BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>>>>> allocate memory for it. Will have maybe have to find another way.
>>>>> qemu-trad those not seems to allocate memory, but I haven't been
>>>>> very far in trying to check that.
>>>>
>>>> And indeed that is the case. The "Fix" below fixes it.
>>>>
>>>>
>>>> Based on that and this guest config:
>>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>>>> memory = 2048
>>>> boot="d"
>>>> maxvcpus=32
>>>> vcpus=1
>>>> serial='pty'
>>>> vnclisten="0.0.0.0"
>>>> name="latest"
>>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ] pci =
>>>> ["01:00.0"]
>>>>
>>>> I can boot the guest.
>>>
>>> And can you access the ROM from the guest ?
>>>
>>>
>>> Also, I have another patch, it will initialize the PCI ROM BAR like
>>> any other BAR. In this case, if qemu is envolved in the access to ROM,
>>> it will print an error, like it the case for other BAR.
>>>
>>> I tried to test it, but it was with an embedded VGA card. When I dump
>>> the ROM, I got the same one as the emulated card instead of the ROM
>>> from the device.
>>>
>>>
>>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index
>>> 6dd7a68..2bbdb6d
>>> 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -440,8
>>> +440,8 @@ static int
>>> xen_pt_register_regions(XenPCIPassthroughState *s)
>>>
>>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>>> - "xen-pci-pt-rom", d->rom.size);
>>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev, +
>>> "xen-pci-pt-rom", d->rom.size);
>>> pci_register_bar(&s->dev, PCI_ROM_SLOT,
>>> PCI_BASE_ADDRESS_MEM_PREFETCH,
>>> &s->rom);
>>
>> Hi, Anthony,
>>
>> Does your fixing is the final solution for this issue? If yes, will
>> you push it
> before Xen 4.4 release?
>
> I included this patch in the last pull request I sent to Anthony Liguori:
>
> http://marc.info/?l=qemu-devel&m=138997319906095
>
> It hasn't been pulled yet, but I would expect that it is going to be
> upstream soon.
> Regarding the 4.4 release, we are trying to fix a couple of other
> serious bugs in the qemu-xen tree right now, but it is still
> conceivable to have this fix backported to the tree in time for the release.
Hope it can catch the 4.4 release.
BTW: Do you know when Xen 4.4 will be released? I saw the wiki documented that it should be Jan 21. But it seems there still will have RC3 come out on Feb 9.
Best regards,
Yang
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-27 15:34 ` Zhang, Yang Z
@ 2014-01-27 15:37 ` Stefano Stabellini
2014-01-31 15:23 ` Stefano Stabellini
1 sibling, 0 replies; 39+ messages in thread
From: Stefano Stabellini @ 2014-01-27 15:37 UTC (permalink / raw)
To: Zhang, Yang Z
Cc: Stefano Stabellini, Dugger, Donald D, stefano.stabellini,
Anthony PERARD, xen-devel
On Mon, 27 Jan 2014, Zhang, Yang Z wrote:
> Stefano Stabellini wrote on 2014-01-27:
> > On Sun, 26 Jan 2014, Zhang, Yang Z wrote:
> >> Anthony PERARD wrote on 2014-01-09:
> >>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> >>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> >>>>> On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk
> > wrote:
> >>>>>> On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> >>>>>>> On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> >>>>>>> [...]
> >>>>>>>>> Those Xen report something like:
> >>>>>>>>> (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46:
> >>>>>>>>> 131329 >
> >>>>>>>>> 131328
> >>>>>>>>> (XEN) memory.c:132:d0 Could not allocate order=0 extent:
> >>>>>>>>> id=46
> >>>>>>>>> memflags=0 (62 of 64)
> >>>>>>>>>
> >>>>>>>>> ?
> >>>>>>>>>
> >>>>>>>>> (I tryied to reproduce the issue by simply add many emulated
> >>>>>>>>> e1000 in QEMU :) )
> >>>>>>>>>
> >>>>>
> >>>>>> -bash-4.1# lspci -s 01:00.0 -v
> >>>>>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> >>>>>> Network
> >>> Connection (rev 01)
> >>>>>> Subsystem: Intel Corporation Gigabit ET Dual Port Server
> >>>>>> Adapter Flags: fast devsel, IRQ 16 Memory at fbc20000
> >>>>>> (32-bit, non-prefetchable) [disabled] [size=128K] Memory at
> >>>>>> fb800000 (32-bit, non-prefetchable) [disabled] [size=4M] I/O
> >>>>>> ports at e020 [disabled] [size=32] Memory at fbc44000
> >>>>>> (32-bit, non-prefetchable) [disabled] [size=16K] Expansion
> >>>>>> ROM at fb400000 [disabled] [size=4M]
> >>>>>
> >>>>> BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> >>>>> allocate memory for it. Will have maybe have to find another way.
> >>>>> qemu-trad those not seems to allocate memory, but I haven't been
> >>>>> very far in trying to check that.
> >>>>
> >>>> And indeed that is the case. The "Fix" below fixes it.
> >>>>
> >>>>
> >>>> Based on that and this guest config:
> >>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> >>>> memory = 2048
> >>>> boot="d"
> >>>> maxvcpus=32
> >>>> vcpus=1
> >>>> serial='pty'
> >>>> vnclisten="0.0.0.0"
> >>>> name="latest"
> >>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ] pci =
> >>>> ["01:00.0"]
> >>>>
> >>>> I can boot the guest.
> >>>
> >>> And can you access the ROM from the guest ?
> >>>
> >>>
> >>> Also, I have another patch, it will initialize the PCI ROM BAR like
> >>> any other BAR. In this case, if qemu is envolved in the access to ROM,
> >>> it will print an error, like it the case for other BAR.
> >>>
> >>> I tried to test it, but it was with an embedded VGA card. When I dump
> >>> the ROM, I got the same one as the emulated card instead of the ROM
> >>> from the device.
> >>>
> >>>
> >>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index
> >>> 6dd7a68..2bbdb6d
> >>> 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -440,8
> >>> +440,8 @@ static int
> >>> xen_pt_register_regions(XenPCIPassthroughState *s)
> >>>
> >>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> >>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> >>> - "xen-pci-pt-rom", d->rom.size);
> >>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev, +
> >>> "xen-pci-pt-rom", d->rom.size);
> >>> pci_register_bar(&s->dev, PCI_ROM_SLOT,
> >>> PCI_BASE_ADDRESS_MEM_PREFETCH,
> >>> &s->rom);
> >>
> >> Hi, Anthony,
> >>
> >> Does your fixing is the final solution for this issue? If yes, will
> >> you push it
> > before Xen 4.4 release?
> >
> > I included this patch in the last pull request I sent to Anthony Liguori:
> >
> > http://marc.info/?l=qemu-devel&m=138997319906095
> >
> > It hasn't been pulled yet, but I would expect that it is going to be
> > upstream soon.
> > Regarding the 4.4 release, we are trying to fix a couple of other
> > serious bugs in the qemu-xen tree right now, but it is still
> > conceivable to have this fix backported to the tree in time for the release.
>
> Hope it can catch the 4.4 release.
>
> BTW: Do you know when Xen 4.4 will be released? I saw the wiki documented that it should be Jan 21. But it seems there still will have RC3 come out on Feb 9.
When it is ready ;-)
It is probably going to be "soon".
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-27 15:34 ` Zhang, Yang Z
2014-01-27 15:37 ` Stefano Stabellini
@ 2014-01-31 15:23 ` Stefano Stabellini
1 sibling, 0 replies; 39+ messages in thread
From: Stefano Stabellini @ 2014-01-31 15:23 UTC (permalink / raw)
To: Zhang, Yang Z
Cc: Stefano Stabellini, Dugger, Donald D, stefano.stabellini,
Anthony PERARD, xen-devel
On Mon, 27 Jan 2014, Zhang, Yang Z wrote:
> Stefano Stabellini wrote on 2014-01-27:
> > On Sun, 26 Jan 2014, Zhang, Yang Z wrote:
> >> Anthony PERARD wrote on 2014-01-09:
> >>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
> >>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
> >>>>> On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk
> > wrote:
> >>>>>> On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
> >>>>>>> On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
> >>>>>>> [...]
> >>>>>>>>> Those Xen report something like:
> >>>>>>>>> (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46:
> >>>>>>>>> 131329 >
> >>>>>>>>> 131328
> >>>>>>>>> (XEN) memory.c:132:d0 Could not allocate order=0 extent:
> >>>>>>>>> id=46
> >>>>>>>>> memflags=0 (62 of 64)
> >>>>>>>>>
> >>>>>>>>> ?
> >>>>>>>>>
> >>>>>>>>> (I tryied to reproduce the issue by simply add many emulated
> >>>>>>>>> e1000 in QEMU :) )
> >>>>>>>>>
> >>>>>
> >>>>>> -bash-4.1# lspci -s 01:00.0 -v
> >>>>>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> >>>>>> Network
> >>> Connection (rev 01)
> >>>>>> Subsystem: Intel Corporation Gigabit ET Dual Port Server
> >>>>>> Adapter Flags: fast devsel, IRQ 16 Memory at fbc20000
> >>>>>> (32-bit, non-prefetchable) [disabled] [size=128K] Memory at
> >>>>>> fb800000 (32-bit, non-prefetchable) [disabled] [size=4M] I/O
> >>>>>> ports at e020 [disabled] [size=32] Memory at fbc44000
> >>>>>> (32-bit, non-prefetchable) [disabled] [size=16K] Expansion
> >>>>>> ROM at fb400000 [disabled] [size=4M]
> >>>>>
> >>>>> BTW, I think this is the issue, the Expansion ROM. qemu-xen will
> >>>>> allocate memory for it. Will have maybe have to find another way.
> >>>>> qemu-trad those not seems to allocate memory, but I haven't been
> >>>>> very far in trying to check that.
> >>>>
> >>>> And indeed that is the case. The "Fix" below fixes it.
> >>>>
> >>>>
> >>>> Based on that and this guest config:
> >>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> >>>> memory = 2048
> >>>> boot="d"
> >>>> maxvcpus=32
> >>>> vcpus=1
> >>>> serial='pty'
> >>>> vnclisten="0.0.0.0"
> >>>> name="latest"
> >>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ] pci =
> >>>> ["01:00.0"]
> >>>>
> >>>> I can boot the guest.
> >>>
> >>> And can you access the ROM from the guest ?
> >>>
> >>>
> >>> Also, I have another patch, it will initialize the PCI ROM BAR like
> >>> any other BAR. In this case, if qemu is envolved in the access to ROM,
> >>> it will print an error, like it the case for other BAR.
> >>>
> >>> I tried to test it, but it was with an embedded VGA card. When I dump
> >>> the ROM, I got the same one as the emulated card instead of the ROM
> >>> from the device.
> >>>
> >>>
> >>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index
> >>> 6dd7a68..2bbdb6d
> >>> 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -440,8
> >>> +440,8 @@ static int
> >>> xen_pt_register_regions(XenPCIPassthroughState *s)
> >>>
> >>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
> >>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> >>> - "xen-pci-pt-rom", d->rom.size);
> >>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev, +
> >>> "xen-pci-pt-rom", d->rom.size);
> >>> pci_register_bar(&s->dev, PCI_ROM_SLOT,
> >>> PCI_BASE_ADDRESS_MEM_PREFETCH,
> >>> &s->rom);
> >>
> >> Hi, Anthony,
> >>
> >> Does your fixing is the final solution for this issue? If yes, will
> >> you push it
> > before Xen 4.4 release?
> >
> > I included this patch in the last pull request I sent to Anthony Liguori:
> >
> > http://marc.info/?l=qemu-devel&m=138997319906095
> >
> > It hasn't been pulled yet, but I would expect that it is going to be
> > upstream soon.
> > Regarding the 4.4 release, we are trying to fix a couple of other
> > serious bugs in the qemu-xen tree right now, but it is still
> > conceivable to have this fix backported to the tree in time for the release.
>
> Hope it can catch the 4.4 release.
The fix is in qemu-xen now.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: qemu-xen-dir + PCI passthrough = BOOM
2014-01-09 14:56 ` Anthony PERARD
` (3 preceding siblings ...)
2014-01-26 0:05 ` Zhang, Yang Z
@ 2014-01-26 2:15 ` Zhang, Yang Z
4 siblings, 0 replies; 39+ messages in thread
From: Zhang, Yang Z @ 2014-01-26 2:15 UTC (permalink / raw)
To: Anthony PERARD, Konrad Rzeszutek Wilk
Cc: xen-devel, stefano.stabellini, Liu, SongtaoX, Dugger, Donald D
Zhang, Yang Z wrote on 2014-01-26:
> Anthony PERARD wrote on 2014-01-09:
>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>>>> On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>>>>> On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>>>>>> On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>>>>>> [...]
>>>>>>>> Those Xen report something like:
>>>>>>>> (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46:
>>>>>>>> 131329 >
>>>>>>>> 131328
>>>>>>>> (XEN) memory.c:132:d0 Could not allocate order=0 extent:
>>>>>>>> id=46
>>>>>>>> memflags=0 (62 of 64)
>>>>>>>>
>>>>>>>> ?
>>>>>>>>
>>>>>>>> (I tryied to reproduce the issue by simply add many emulated
>>>>>>>> e1000 in QEMU :) )
>>>>>>>>
>>>>
>>>>> -bash-4.1# lspci -s 01:00.0 -v
>>>>> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
>>>>> Network
>> Connection (rev 01)
>>>>> Subsystem: Intel Corporation Gigabit ET Dual Port Server
>>>>> Adapter Flags: fast devsel, IRQ 16 Memory at fbc20000
>>>>> (32-bit, non-prefetchable) [disabled] [size=128K] Memory at
>>>>> fb800000 (32-bit, non-prefetchable) [disabled] [size=4M] I/O
>>>>> ports at e020 [disabled] [size=32] Memory at fbc44000
>>>>> (32-bit, non-prefetchable) [disabled] [size=16K] Expansion
>>>>> ROM at fb400000 [disabled] [size=4M]
>>>>
>>>> BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>>>> allocate memory for it. Will have maybe have to find another way.
>>>> qemu-trad those not seems to allocate memory, but I haven't been
>>>> very far in trying to check that.
>>>
>>> And indeed that is the case. The "Fix" below fixes it.
>>>
>>>
>>> Based on that and this guest config:
>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>>> memory = 2048
>>> boot="d"
>>> maxvcpus=32
>>> vcpus=1
>>> serial='pty'
>>> vnclisten="0.0.0.0"
>>> name="latest"
>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ] pci = ["01:00.0"]
>>>
>>> I can boot the guest.
>>
>> And can you access the ROM from the guest ?
>>
>>
>> Also, I have another patch, it will initialize the PCI ROM BAR like any
>> other BAR. In this case, if qemu is envolved in the access to ROM, it
>> will print an error, like it the case for other BAR.
>>
>> I tried to test it, but it was with an embedded VGA card. When I dump
>> the ROM, I got the same one as the emulated card instead of the ROM
>> from the device.
>>
>>
>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c index 6dd7a68..2bbdb6d
>> 100644 --- a/hw/xen/xen_pt.c +++ b/hw/xen/xen_pt.c @@ -440,8 +440,8 @@
>> static int xen_pt_register_regions(XenPCIPassthroughState *s)
>>
>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>> - "xen-pci-pt-rom", d->rom.size);
>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev, +
>> "xen-pci-pt-rom", d->rom.size);
>> pci_register_bar(&s->dev, PCI_ROM_SLOT,
>> PCI_BASE_ADDRESS_MEM_PREFETCH,
>> &s->rom);
>
> Hi, Anthony,
>
> Does your fixing is the final solution for this issue? If yes, will
> you push it before Xen 4.4 release?
>
> Best regards,
> Yang
>
CC songtao who still saw this issue with qemu-xen. If the patch is in, please let us know. We hope it will be included in Xen 4.4 release.
Best regards,
Yang
^ permalink raw reply [flat|nested] 39+ messages in thread