* xl pci-attach silently fails the first time
@ 2014-12-01 12:57 Olaf Hering
2014-12-01 13:32 ` Olaf Hering
0 siblings, 1 reply; 18+ messages in thread
From: Olaf Hering @ 2014-12-01 12:57 UTC (permalink / raw)
To: xen-devel
I tried to attach a PCI device (IGB Virtual Function) to a HVM guest.
To actually get it assigned its required to run
pci-attach/pci-detach/pci-attach because it does not show up right away.
Did anyone noticed this bug already, is there a fix? There is no error
reported in dom0 dmesg, xl dmesg or qemu logfiles.
My domU.cfg looks like this:
name="domU"
memory=512
builder="hvm"
vif=['',]
disk=[ 'file:/SLE-12-Server-DVD-x86_64-GM-DVD1.iso,hda:cdrom' ]
serial="pty"
# xl pci-assignable-add 01:10.0
# xl pci-assignable-list
0000:01:10.0
# xl create -f domU.cfg
# xl console domU
## lspci gives just emulated PCI devices
## detach from domU console
# xl pci-attach domU 0000:01:10.0
# xl pci-list domU
Vdev Device
04.0 0000:01:10.0
# xl console domU
## lspci gives just emulated PCI devices
## detach from domU console
# xl pci-detach domU 0000:01:10.0
# xl pci-attach domU 0000:01:10.0
# xl pci-list domU
Vdev Device
04.0 0000:01:10.0
# xl console domU
## lspci shows now also the assigned host device
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 12:57 xl pci-attach silently fails the first time Olaf Hering
@ 2014-12-01 13:32 ` Olaf Hering
2014-12-01 13:42 ` Olaf Hering
2014-12-01 17:01 ` Konrad Rzeszutek Wilk
0 siblings, 2 replies; 18+ messages in thread
From: Olaf Hering @ 2014-12-01 13:32 UTC (permalink / raw)
To: xen-devel
On Mon, Dec 01, Olaf Hering wrote:
> # xl pci-assignable-add 01:10.0
> # xl pci-assignable-list
> 0000:01:10.0
> # xl create -f domU.cfg
> # xl console domU
> ## lspci gives just emulated PCI devices
ttyS0:Rescue:~ # lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff)
> ## detach from domU console
> # xl pci-attach domU 0000:01:10.0
> # xl pci-list domU
> Vdev Device
> 04.0 0000:01:10.0
>
> # xl console domU
> ## lspci gives just emulated PCI devices
ttyS0:Rescue:~ # lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01)
> ## detach from domU console
> # xl pci-detach domU 0000:01:10.0
Now lspci shows that the emulated network card is gone.
ttyS0:Rescue:~ # lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> # xl pci-attach domU 0000:01:10.0
> # xl pci-list domU
> Vdev Device
> 04.0 0000:01:10.0
> # xl console domU
> ## lspci shows now also the assigned host device
So the actual bug is that the very first time after pci-attach the guests
"00:04.0" PCI device is (most likely) replaced with the host PCI device. Just
the guest does not notice that "00:04.0" was actually already gone after unplug.
I wonder why the unplug code in xen_platform.c does just a qdev_free() instead
of a real pci-detach kind of thing. I'm sure this could be done at least for
emulated network cards.
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 13:32 ` Olaf Hering
@ 2014-12-01 13:42 ` Olaf Hering
2014-12-01 13:57 ` Sander Eikelenboom
2014-12-01 17:01 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 18+ messages in thread
From: Olaf Hering @ 2014-12-01 13:42 UTC (permalink / raw)
To: xen-devel
On Mon, Dec 01, Olaf Hering wrote:
> > # xl pci-attach domU 0000:01:10.0
"xl pci-attach -h" mentions [Virtual Slot], but the code does not seem
to handle an additional parameter, pciattach() ignores *vs.
What is the "Virtual Slot", why is it ignored?
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 13:42 ` Olaf Hering
@ 2014-12-01 13:57 ` Sander Eikelenboom
2014-12-01 14:34 ` Olaf Hering
0 siblings, 1 reply; 18+ messages in thread
From: Sander Eikelenboom @ 2014-12-01 13:57 UTC (permalink / raw)
To: Olaf Hering; +Cc: xen-devel
Monday, December 1, 2014, 2:42:11 PM, you wrote:
> On Mon, Dec 01, Olaf Hering wrote:
>> > # xl pci-attach domU 0000:01:10.0
> "xl pci-attach -h" mentions [Virtual Slot], but the code does not seem
> to handle an additional parameter, pciattach() ignores *vs.
> What is the "Virtual Slot", why is it ignored?
Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough
It was the ability with xend + qemu-trad to be able to specify the slot to
use in the guest for the pci device.
See docs/misc/vtd.txt .. that seems it has never been updated :-)
(together with passing through a multifunction devices as multifunction inside
the guest this hasn't got implemented in neither libxl and qemu-xen.)
--
Sander
>From docs/misc/vtd.txt:
VTd device hotplug:
-------------------
2 virtual PCI slots (6~7) are reserved in HVM guest to support VTd hotplug. If you have more VTd devices, only 2 of them can support hotplug. Usage is simple:
1. List the VTd device by dom. You can see a VTd device 0:2:0.0 is inserted in the HVM domain's PCI slot 6. '''lspci''' inside the guest should see the same.
[root@vt-vtd ~]# xm pci-list HVMDomainVtd
VSlt domain bus slot func
0x6 0x0 0x02 0x00 0x0
2. Detach the device from the guest by the physical BDF. Then HVM guest will receive a virtual PCI hot removal event to detach the physical device
[root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0
3. Attach a PCI device to the guest by the physical BDF and desired virtual slot(optional). Following command would insert the physical device into guest's virtual slot 7
[root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7
To specify options for the device, use -o or --options=. Following command would disable MSI-INTx translation for the device
[root@vt-vtd ~]# xm pci-attach -o msitranslate=0 0:2:0.0 7
> Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 13:57 ` Sander Eikelenboom
@ 2014-12-01 14:34 ` Olaf Hering
2014-12-01 14:39 ` Ian Campbell
2014-12-01 14:46 ` Sander Eikelenboom
0 siblings, 2 replies; 18+ messages in thread
From: Olaf Hering @ 2014-12-01 14:34 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: xen-devel
On Mon, Dec 01, Sander Eikelenboom wrote:
> Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough
>
> It was the ability with xend + qemu-trad to be able to specify the slot to
> use in the guest for the pci device.
>
> See docs/misc/vtd.txt .. that seems it has never been updated :-)
>
> (together with passing through a multifunction devices as multifunction inside
> the guest this hasn't got implemented in neither libxl and qemu-xen.)
So this looks like we a lost feature in libxl. Not sure if that would
actually be a workaround for the double pci-attach bug.
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 14:34 ` Olaf Hering
@ 2014-12-01 14:39 ` Ian Campbell
2014-12-01 14:46 ` Olaf Hering
2014-12-01 14:46 ` Sander Eikelenboom
1 sibling, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2014-12-01 14:39 UTC (permalink / raw)
To: Olaf Hering; +Cc: Sander Eikelenboom, xen-devel
On Mon, 2014-12-01 at 15:34 +0100, Olaf Hering wrote:
> On Mon, Dec 01, Sander Eikelenboom wrote:
>
> > Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough
> >
> > It was the ability with xend + qemu-trad to be able to specify the slot to
> > use in the guest for the pci device.
> >
> > See docs/misc/vtd.txt .. that seems it has never been updated :-)
> >
> > (together with passing through a multifunction devices as multifunction inside
> > the guest this hasn't got implemented in neither libxl and qemu-xen.)
>
> So this looks like we a lost feature in libxl.
See http://bugs.xenproject.org/xen/bug/22 (I think that's what that is)
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 14:39 ` Ian Campbell
@ 2014-12-01 14:46 ` Olaf Hering
0 siblings, 0 replies; 18+ messages in thread
From: Olaf Hering @ 2014-12-01 14:46 UTC (permalink / raw)
To: Ian Campbell; +Cc: Sander Eikelenboom, xen-devel
On Mon, Dec 01, Ian Campbell wrote:
> On Mon, 2014-12-01 at 15:34 +0100, Olaf Hering wrote:
> > On Mon, Dec 01, Sander Eikelenboom wrote:
> > > Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough
> > >
> > > It was the ability with xend + qemu-trad to be able to specify the slot to
> > > use in the guest for the pci device.
> > >
> > > See docs/misc/vtd.txt .. that seems it has never been updated :-)
> > >
> > > (together with passing through a multifunction devices as multifunction inside
> > > the guest this hasn't got implemented in neither libxl and qemu-xen.)
> > So this looks like we a lost feature in libxl.
> See http://bugs.xenproject.org/xen/bug/22 (I think that's what that is)
Yes, the virtual function part is covered by #22. Thanks.
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 14:34 ` Olaf Hering
2014-12-01 14:39 ` Ian Campbell
@ 2014-12-01 14:46 ` Sander Eikelenboom
2014-12-02 15:47 ` Olaf Hering
1 sibling, 1 reply; 18+ messages in thread
From: Sander Eikelenboom @ 2014-12-01 14:46 UTC (permalink / raw)
To: Olaf Hering; +Cc: xen-devel
Monday, December 1, 2014, 3:34:09 PM, you wrote:
> On Mon, Dec 01, Sander Eikelenboom wrote:
>> Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough
>>
>> It was the ability with xend + qemu-trad to be able to specify the slot to
>> use in the guest for the pci device.
>>
>> See docs/misc/vtd.txt .. that seems it has never been updated :-)
>>
>> (together with passing through a multifunction devices as multifunction inside
>> the guest this hasn't got implemented in neither libxl and qemu-xen.)
> So this looks like we a lost feature in libxl.
Yes, they are on the bug list though for quite some time :-).
see http://bugs.xenproject.org/xen/bug/22
Although the multifuction part missing is only slightly noted there.
(a fix would need a part libxl and for HVM's at least part qemu-xen
(since qemu-xen doesn't have the option implemented that qemu-trad
has for the slot and multifunction assignment))
> actually be a workaround for the double pci-attach bug.
Don't know about that bug.
--
Sander
> Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 13:32 ` Olaf Hering
2014-12-01 13:42 ` Olaf Hering
@ 2014-12-01 17:01 ` Konrad Rzeszutek Wilk
2014-12-02 15:46 ` Olaf Hering
2014-12-04 1:31 ` Konrad Rzeszutek Wilk
1 sibling, 2 replies; 18+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-12-01 17:01 UTC (permalink / raw)
To: Olaf Hering; +Cc: xen-devel
On Mon, Dec 01, 2014 at 02:32:44PM +0100, Olaf Hering wrote:
> On Mon, Dec 01, Olaf Hering wrote:
>
> > # xl pci-assignable-add 01:10.0
> > # xl pci-assignable-list
> > 0000:01:10.0
> > # xl create -f domU.cfg
> > # xl console domU
> > ## lspci gives just emulated PCI devices
>
> ttyS0:Rescue:~ # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff)
>
> > ## detach from domU console
> > # xl pci-attach domU 0000:01:10.0
> > # xl pci-list domU
> > Vdev Device
> > 04.0 0000:01:10.0
> >
> > # xl console domU
> > ## lspci gives just emulated PCI devices
>
> ttyS0:Rescue:~ # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01)
>
> > ## detach from domU console
> > # xl pci-detach domU 0000:01:10.0
> Now lspci shows that the emulated network card is gone.
> ttyS0:Rescue:~ # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
>
> > # xl pci-attach domU 0000:01:10.0
> > # xl pci-list domU
> > Vdev Device
> > 04.0 0000:01:10.0
> > # xl console domU
> > ## lspci shows now also the assigned host device
>
>
> So the actual bug is that the very first time after pci-attach the guests
> "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just
> the guest does not notice that "00:04.0" was actually already gone after unplug.
That is odd - I see any device 'hot-plugged' being added at 00:05 and further.
>
> I wonder why the unplug code in xen_platform.c does just a qdev_free() instead
> of a real pci-detach kind of thing. I'm sure this could be done at least for
> emulated network cards.
>
> Olaf
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 17:01 ` Konrad Rzeszutek Wilk
@ 2014-12-02 15:46 ` Olaf Hering
2014-12-02 18:39 ` Konrad Rzeszutek Wilk
2014-12-04 1:31 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 18+ messages in thread
From: Olaf Hering @ 2014-12-02 15:46 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
On Mon, Dec 01, Konrad Rzeszutek Wilk wrote:
> That is odd - I see any device 'hot-plugged' being added at 00:05 and further.
Does this by any chance depend on the guest?! I mean, how is the guest
notified that a PCI device is gone (by unplug)? Maybe the pvops case
just happens to work because the unplug happens early, perhaps before
PCI discovery?!
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 14:46 ` Sander Eikelenboom
@ 2014-12-02 15:47 ` Olaf Hering
0 siblings, 0 replies; 18+ messages in thread
From: Olaf Hering @ 2014-12-02 15:47 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: xen-devel
On Mon, Dec 01, Sander Eikelenboom wrote:
> Monday, December 1, 2014, 3:34:09 PM, you wrote:
> > actually be a workaround for the double pci-attach bug.
> Don't know about that bug.
You just replied to it. ;-)
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-02 15:46 ` Olaf Hering
@ 2014-12-02 18:39 ` Konrad Rzeszutek Wilk
2014-12-03 8:36 ` Olaf Hering
0 siblings, 1 reply; 18+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-12-02 18:39 UTC (permalink / raw)
To: Olaf Hering; +Cc: xen-devel
On Tue, Dec 02, 2014 at 04:46:52PM +0100, Olaf Hering wrote:
> On Mon, Dec 01, Konrad Rzeszutek Wilk wrote:
>
> > That is odd - I see any device 'hot-plugged' being added at 00:05 and further.
>
> Does this by any chance depend on the guest?! I mean, how is the guest
I doubt it.
> notified that a PCI device is gone (by unplug)? Maybe the pvops case
> just happens to work because the unplug happens early, perhaps before
> PCI discovery?!
ACPI hotplug. And it does work after PCI discovery.
>
> Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-02 18:39 ` Konrad Rzeszutek Wilk
@ 2014-12-03 8:36 ` Olaf Hering
2014-12-03 11:24 ` Olaf Hering
0 siblings, 1 reply; 18+ messages in thread
From: Olaf Hering @ 2014-12-03 8:36 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
On Tue, Dec 02, Konrad Rzeszutek Wilk wrote:
> On Tue, Dec 02, 2014 at 04:46:52PM +0100, Olaf Hering wrote:
> > On Mon, Dec 01, Konrad Rzeszutek Wilk wrote:
> >
> > > That is odd - I see any device 'hot-plugged' being added at 00:05 and further.
> >
> > Does this by any chance depend on the guest?! I mean, how is the guest
>
> I doubt it.
Something is different here, likely just the non-pvops guest kernel.
> > notified that a PCI device is gone (by unplug)? Maybe the pvops case
> > just happens to work because the unplug happens early, perhaps before
> > PCI discovery?!
>
> ACPI hotplug. And it does work after PCI discovery.
In a pvops kernel, is the emulated but unplugged PCI hardware still listed with lspci?
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-03 8:36 ` Olaf Hering
@ 2014-12-03 11:24 ` Olaf Hering
0 siblings, 0 replies; 18+ messages in thread
From: Olaf Hering @ 2014-12-03 11:24 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
On Wed, Dec 03, Olaf Hering wrote:
> On Tue, Dec 02, Konrad Rzeszutek Wilk wrote:
> > On Tue, Dec 02, 2014 at 04:46:52PM +0100, Olaf Hering wrote:
> > > On Mon, Dec 01, Konrad Rzeszutek Wilk wrote:
> > ACPI hotplug. And it does work after PCI discovery.
> In a pvops kernel, is the emulated but unplugged PCI hardware still listed with lspci?
It is not. So thats why it happens to work.
So how would I trigger an ACPI hotplug event within qemus unplug code?
Olaf
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-01 17:01 ` Konrad Rzeszutek Wilk
2014-12-02 15:46 ` Olaf Hering
@ 2014-12-04 1:31 ` Konrad Rzeszutek Wilk
2014-12-04 1:46 ` Konrad Rzeszutek Wilk
2014-12-04 9:28 ` Ian Campbell
1 sibling, 2 replies; 18+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-12-04 1:31 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Olaf Hering, xen-devel
On Mon, Dec 01, 2014 at 12:01:51PM -0500, Konrad Rzeszutek Wilk wrote:
> On Mon, Dec 01, 2014 at 02:32:44PM +0100, Olaf Hering wrote:
> > On Mon, Dec 01, Olaf Hering wrote:
> >
> > > # xl pci-assignable-add 01:10.0
> > > # xl pci-assignable-list
> > > 0000:01:10.0
> > > # xl create -f domU.cfg
> > > # xl console domU
> > > ## lspci gives just emulated PCI devices
> >
> > ttyS0:Rescue:~ # lspci
> > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff)
> >
> > > ## detach from domU console
> > > # xl pci-attach domU 0000:01:10.0
> > > # xl pci-list domU
> > > Vdev Device
> > > 04.0 0000:01:10.0
> > >
> > > # xl console domU
> > > ## lspci gives just emulated PCI devices
> >
> > ttyS0:Rescue:~ # lspci
> > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01)
> >
> > > ## detach from domU console
> > > # xl pci-detach domU 0000:01:10.0
> > Now lspci shows that the emulated network card is gone.
>
> > ttyS0:Rescue:~ # lspci
> > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> >
> > > # xl pci-attach domU 0000:01:10.0
> > > # xl pci-list domU
> > > Vdev Device
> > > 04.0 0000:01:10.0
> > > # xl console domU
> > > ## lspci shows now also the assigned host device
> >
> >
> > So the actual bug is that the very first time after pci-attach the guests
> > "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just
> > the guest does not notice that "00:04.0" was actually already gone after unplug.
>
> That is odd - I see any device 'hot-plugged' being added at 00:05 and further.
I have to apologize. The reason it works for me is because the emulated
device gets unplugged quite early:
# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
[and here I run 'xl pci-attach USB 00:1a.0]
# [ 30.609802] hpet1: lost 1589 rtc interrupts
[ 30.672030] hpet1: lost 2200 rtc interrupts
[ 30.672030] hpet1: lost 2201 rtc interrupts
[ 30.672030] hpet1: lost 2200 rtc interrupts
[ 30.899341] pci 0000:00:04.0: [8086:1c2d] type 00 class 0x0c0320
And the other test I have been running is when the guest is booted
with an PCI device (and then unplugged):
# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
00:05.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04)
But oddly enough when I do 'pci-detach' and then 'pci-attach' I get:
lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
00:04.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04)
so something is busted as you surmised
And it looks to be busted in two cases - I see the 'hpet1' issues
and the guest then crashes!
>
> >
> > I wonder why the unplug code in xen_platform.c does just a qdev_free() instead
> > of a real pci-detach kind of thing. I'm sure this could be done at least for
> > emulated network cards.
> >
> > Olaf
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-04 1:31 ` Konrad Rzeszutek Wilk
@ 2014-12-04 1:46 ` Konrad Rzeszutek Wilk
2014-12-04 9:28 ` Ian Campbell
1 sibling, 0 replies; 18+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-12-04 1:46 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Olaf Hering, xen-devel
On Wed, Dec 03, 2014 at 09:31:03PM -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Dec 01, 2014 at 12:01:51PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Mon, Dec 01, 2014 at 02:32:44PM +0100, Olaf Hering wrote:
> > > On Mon, Dec 01, Olaf Hering wrote:
> > >
> > > > # xl pci-assignable-add 01:10.0
> > > > # xl pci-assignable-list
> > > > 0000:01:10.0
> > > > # xl create -f domU.cfg
> > > > # xl console domU
> > > > ## lspci gives just emulated PCI devices
> > >
> > > ttyS0:Rescue:~ # lspci
> > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> > > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff)
> > >
> > > > ## detach from domU console
> > > > # xl pci-attach domU 0000:01:10.0
> > > > # xl pci-list domU
> > > > Vdev Device
> > > > 04.0 0000:01:10.0
> > > >
> > > > # xl console domU
> > > > ## lspci gives just emulated PCI devices
> > >
> > > ttyS0:Rescue:~ # lspci
> > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> > > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01)
> > >
> > > > ## detach from domU console
> > > > # xl pci-detach domU 0000:01:10.0
> > > Now lspci shows that the emulated network card is gone.
> >
> > > ttyS0:Rescue:~ # lspci
> > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
> > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> > >
> > > > # xl pci-attach domU 0000:01:10.0
> > > > # xl pci-list domU
> > > > Vdev Device
> > > > 04.0 0000:01:10.0
> > > > # xl console domU
> > > > ## lspci shows now also the assigned host device
> > >
> > >
> > > So the actual bug is that the very first time after pci-attach the guests
> > > "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just
> > > the guest does not notice that "00:04.0" was actually already gone after unplug.
> >
> > That is odd - I see any device 'hot-plugged' being added at 00:05 and further.
>
> I have to apologize. The reason it works for me is because the emulated
> device gets unplugged quite early:
>
> # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
>
> [and here I run 'xl pci-attach USB 00:1a.0]
>
> # [ 30.609802] hpet1: lost 1589 rtc interrupts
> [ 30.672030] hpet1: lost 2200 rtc interrupts
> [ 30.672030] hpet1: lost 2201 rtc interrupts
> [ 30.672030] hpet1: lost 2200 rtc interrupts
> [ 30.899341] pci 0000:00:04.0: [8086:1c2d] type 00 class 0x0c0320
>
> And the other test I have been running is when the guest is booted
> with an PCI device (and then unplugged):
>
> # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
> 00:02.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> 00:05.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04)
>
> But oddly enough when I do 'pci-detach' and then 'pci-attach' I get:
> lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
> 00:02.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> 00:04.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04)
>
> so something is busted as you surmised
>
> And it looks to be busted in two cases - I see the 'hpet1' issues
> and the guest then crashes!
But that only shows up on this particular machine. If I use VFs and
unplug/plug it works OK.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-04 1:31 ` Konrad Rzeszutek Wilk
2014-12-04 1:46 ` Konrad Rzeszutek Wilk
@ 2014-12-04 9:28 ` Ian Campbell
2014-12-04 12:03 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 18+ messages in thread
From: Ian Campbell @ 2014-12-04 9:28 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Olaf Hering, xen-devel
On Wed, 2014-12-03 at 21:31 -0400, Konrad Rzeszutek Wilk wrote:
> so something is busted as you surmised
I haven't been following closely, so this may be completely unrelated to
the issue under discussion, but doesn't in-HVM-guest hotplug require a
guest side driver to be loaded? I recall in the dim and distant past
having to manually load some sort of foo_hp.ko.
Ian.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time
2014-12-04 9:28 ` Ian Campbell
@ 2014-12-04 12:03 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 18+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-12-04 12:03 UTC (permalink / raw)
To: Ian Campbell; +Cc: Olaf Hering, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 777 bytes --]
On Dec 4, 2014 4:29 AM, "Ian Campbell" <Ian.Campbell@citrix.com> wrote:
>
> On Wed, 2014-12-03 at 21:31 -0400, Konrad Rzeszutek Wilk wrote:
> > so something is busted as you surmised
>
> I haven't been following closely, so this may be completely unrelated to
> the issue under discussion, but doesn't in-HVM-guest hotplug require a
> guest side driver to be loaded? I recall in the dim and distant past
> having to manually load some sort of foo_hp.ko.
>
The ACPI PCI hotplug machinery does all of that nicely.
As mentioned - it works nicely - except that the BDF that is chosen is at
an wrong slot. In the 'xend' days in recall this working nicely and
sticking the PF at slot 5 and beyond. Though I have no clue how it dealt
with multiple emulated NICs and such.
> Ian.
>
[-- Attachment #1.2: Type: text/html, Size: 1015 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2014-12-04 12:03 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-12-01 12:57 xl pci-attach silently fails the first time Olaf Hering
2014-12-01 13:32 ` Olaf Hering
2014-12-01 13:42 ` Olaf Hering
2014-12-01 13:57 ` Sander Eikelenboom
2014-12-01 14:34 ` Olaf Hering
2014-12-01 14:39 ` Ian Campbell
2014-12-01 14:46 ` Olaf Hering
2014-12-01 14:46 ` Sander Eikelenboom
2014-12-02 15:47 ` Olaf Hering
2014-12-01 17:01 ` Konrad Rzeszutek Wilk
2014-12-02 15:46 ` Olaf Hering
2014-12-02 18:39 ` Konrad Rzeszutek Wilk
2014-12-03 8:36 ` Olaf Hering
2014-12-03 11:24 ` Olaf Hering
2014-12-04 1:31 ` Konrad Rzeszutek Wilk
2014-12-04 1:46 ` Konrad Rzeszutek Wilk
2014-12-04 9:28 ` Ian Campbell
2014-12-04 12:03 ` Konrad Rzeszutek Wilk
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.