* xl pci-attach silently fails the first time @ 2014-12-01 12:57 Olaf Hering 2014-12-01 13:32 ` Olaf Hering 0 siblings, 1 reply; 18+ messages in thread From: Olaf Hering @ 2014-12-01 12:57 UTC (permalink / raw) To: xen-devel I tried to attach a PCI device (IGB Virtual Function) to a HVM guest. To actually get it assigned its required to run pci-attach/pci-detach/pci-attach because it does not show up right away. Did anyone noticed this bug already, is there a fix? There is no error reported in dom0 dmesg, xl dmesg or qemu logfiles. My domU.cfg looks like this: name="domU" memory=512 builder="hvm" vif=['',] disk=[ 'file:/SLE-12-Server-DVD-x86_64-GM-DVD1.iso,hda:cdrom' ] serial="pty" # xl pci-assignable-add 01:10.0 # xl pci-assignable-list 0000:01:10.0 # xl create -f domU.cfg # xl console domU ## lspci gives just emulated PCI devices ## detach from domU console # xl pci-attach domU 0000:01:10.0 # xl pci-list domU Vdev Device 04.0 0000:01:10.0 # xl console domU ## lspci gives just emulated PCI devices ## detach from domU console # xl pci-detach domU 0000:01:10.0 # xl pci-attach domU 0000:01:10.0 # xl pci-list domU Vdev Device 04.0 0000:01:10.0 # xl console domU ## lspci shows now also the assigned host device Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 12:57 xl pci-attach silently fails the first time Olaf Hering @ 2014-12-01 13:32 ` Olaf Hering 2014-12-01 13:42 ` Olaf Hering 2014-12-01 17:01 ` Konrad Rzeszutek Wilk 0 siblings, 2 replies; 18+ messages in thread From: Olaf Hering @ 2014-12-01 13:32 UTC (permalink / raw) To: xen-devel On Mon, Dec 01, Olaf Hering wrote: > # xl pci-assignable-add 01:10.0 > # xl pci-assignable-list > 0000:01:10.0 > # xl create -f domU.cfg > # xl console domU > ## lspci gives just emulated PCI devices ttyS0:Rescue:~ # lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff) > ## detach from domU console > # xl pci-attach domU 0000:01:10.0 > # xl pci-list domU > Vdev Device > 04.0 0000:01:10.0 > > # xl console domU > ## lspci gives just emulated PCI devices ttyS0:Rescue:~ # lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01) > ## detach from domU console > # xl pci-detach domU 0000:01:10.0 Now lspci shows that the emulated network card is gone. ttyS0:Rescue:~ # lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > # xl pci-attach domU 0000:01:10.0 > # xl pci-list domU > Vdev Device > 04.0 0000:01:10.0 > # xl console domU > ## lspci shows now also the assigned host device So the actual bug is that the very first time after pci-attach the guests "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just the guest does not notice that "00:04.0" was actually already gone after unplug. I wonder why the unplug code in xen_platform.c does just a qdev_free() instead of a real pci-detach kind of thing. I'm sure this could be done at least for emulated network cards. Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 13:32 ` Olaf Hering @ 2014-12-01 13:42 ` Olaf Hering 2014-12-01 13:57 ` Sander Eikelenboom 2014-12-01 17:01 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 18+ messages in thread From: Olaf Hering @ 2014-12-01 13:42 UTC (permalink / raw) To: xen-devel On Mon, Dec 01, Olaf Hering wrote: > > # xl pci-attach domU 0000:01:10.0 "xl pci-attach -h" mentions [Virtual Slot], but the code does not seem to handle an additional parameter, pciattach() ignores *vs. What is the "Virtual Slot", why is it ignored? Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 13:42 ` Olaf Hering @ 2014-12-01 13:57 ` Sander Eikelenboom 2014-12-01 14:34 ` Olaf Hering 0 siblings, 1 reply; 18+ messages in thread From: Sander Eikelenboom @ 2014-12-01 13:57 UTC (permalink / raw) To: Olaf Hering; +Cc: xen-devel Monday, December 1, 2014, 2:42:11 PM, you wrote: > On Mon, Dec 01, Olaf Hering wrote: >> > # xl pci-attach domU 0000:01:10.0 > "xl pci-attach -h" mentions [Virtual Slot], but the code does not seem > to handle an additional parameter, pciattach() ignores *vs. > What is the "Virtual Slot", why is it ignored? Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough It was the ability with xend + qemu-trad to be able to specify the slot to use in the guest for the pci device. See docs/misc/vtd.txt .. that seems it has never been updated :-) (together with passing through a multifunction devices as multifunction inside the guest this hasn't got implemented in neither libxl and qemu-xen.) -- Sander >From docs/misc/vtd.txt: VTd device hotplug: ------------------- 2 virtual PCI slots (6~7) are reserved in HVM guest to support VTd hotplug. If you have more VTd devices, only 2 of them can support hotplug. Usage is simple: 1. List the VTd device by dom. You can see a VTd device 0:2:0.0 is inserted in the HVM domain's PCI slot 6. '''lspci''' inside the guest should see the same. [root@vt-vtd ~]# xm pci-list HVMDomainVtd VSlt domain bus slot func 0x6 0x0 0x02 0x00 0x0 2. Detach the device from the guest by the physical BDF. Then HVM guest will receive a virtual PCI hot removal event to detach the physical device [root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0 3. Attach a PCI device to the guest by the physical BDF and desired virtual slot(optional). Following command would insert the physical device into guest's virtual slot 7 [root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7 To specify options for the device, use -o or --options=. Following command would disable MSI-INTx translation for the device [root@vt-vtd ~]# xm pci-attach -o msitranslate=0 0:2:0.0 7 > Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 13:57 ` Sander Eikelenboom @ 2014-12-01 14:34 ` Olaf Hering 2014-12-01 14:39 ` Ian Campbell 2014-12-01 14:46 ` Sander Eikelenboom 0 siblings, 2 replies; 18+ messages in thread From: Olaf Hering @ 2014-12-01 14:34 UTC (permalink / raw) To: Sander Eikelenboom; +Cc: xen-devel On Mon, Dec 01, Sander Eikelenboom wrote: > Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough > > It was the ability with xend + qemu-trad to be able to specify the slot to > use in the guest for the pci device. > > See docs/misc/vtd.txt .. that seems it has never been updated :-) > > (together with passing through a multifunction devices as multifunction inside > the guest this hasn't got implemented in neither libxl and qemu-xen.) So this looks like we a lost feature in libxl. Not sure if that would actually be a workaround for the double pci-attach bug. Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 14:34 ` Olaf Hering @ 2014-12-01 14:39 ` Ian Campbell 2014-12-01 14:46 ` Olaf Hering 2014-12-01 14:46 ` Sander Eikelenboom 1 sibling, 1 reply; 18+ messages in thread From: Ian Campbell @ 2014-12-01 14:39 UTC (permalink / raw) To: Olaf Hering; +Cc: Sander Eikelenboom, xen-devel On Mon, 2014-12-01 at 15:34 +0100, Olaf Hering wrote: > On Mon, Dec 01, Sander Eikelenboom wrote: > > > Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough > > > > It was the ability with xend + qemu-trad to be able to specify the slot to > > use in the guest for the pci device. > > > > See docs/misc/vtd.txt .. that seems it has never been updated :-) > > > > (together with passing through a multifunction devices as multifunction inside > > the guest this hasn't got implemented in neither libxl and qemu-xen.) > > So this looks like we a lost feature in libxl. See http://bugs.xenproject.org/xen/bug/22 (I think that's what that is) Ian. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 14:39 ` Ian Campbell @ 2014-12-01 14:46 ` Olaf Hering 0 siblings, 0 replies; 18+ messages in thread From: Olaf Hering @ 2014-12-01 14:46 UTC (permalink / raw) To: Ian Campbell; +Cc: Sander Eikelenboom, xen-devel On Mon, Dec 01, Ian Campbell wrote: > On Mon, 2014-12-01 at 15:34 +0100, Olaf Hering wrote: > > On Mon, Dec 01, Sander Eikelenboom wrote: > > > Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough > > > > > > It was the ability with xend + qemu-trad to be able to specify the slot to > > > use in the guest for the pci device. > > > > > > See docs/misc/vtd.txt .. that seems it has never been updated :-) > > > > > > (together with passing through a multifunction devices as multifunction inside > > > the guest this hasn't got implemented in neither libxl and qemu-xen.) > > So this looks like we a lost feature in libxl. > See http://bugs.xenproject.org/xen/bug/22 (I think that's what that is) Yes, the virtual function part is covered by #22. Thanks. Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 14:34 ` Olaf Hering 2014-12-01 14:39 ` Ian Campbell @ 2014-12-01 14:46 ` Sander Eikelenboom 2014-12-02 15:47 ` Olaf Hering 1 sibling, 1 reply; 18+ messages in thread From: Sander Eikelenboom @ 2014-12-01 14:46 UTC (permalink / raw) To: Olaf Hering; +Cc: xen-devel Monday, December 1, 2014, 3:34:09 PM, you wrote: > On Mon, Dec 01, Sander Eikelenboom wrote: >> Hmm the wiki also still mentions it: http://wiki.xen.org/wiki/Xen_PCI_Passthrough >> >> It was the ability with xend + qemu-trad to be able to specify the slot to >> use in the guest for the pci device. >> >> See docs/misc/vtd.txt .. that seems it has never been updated :-) >> >> (together with passing through a multifunction devices as multifunction inside >> the guest this hasn't got implemented in neither libxl and qemu-xen.) > So this looks like we a lost feature in libxl. Yes, they are on the bug list though for quite some time :-). see http://bugs.xenproject.org/xen/bug/22 Although the multifuction part missing is only slightly noted there. (a fix would need a part libxl and for HVM's at least part qemu-xen (since qemu-xen doesn't have the option implemented that qemu-trad has for the slot and multifunction assignment)) > actually be a workaround for the double pci-attach bug. Don't know about that bug. -- Sander > Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 14:46 ` Sander Eikelenboom @ 2014-12-02 15:47 ` Olaf Hering 0 siblings, 0 replies; 18+ messages in thread From: Olaf Hering @ 2014-12-02 15:47 UTC (permalink / raw) To: Sander Eikelenboom; +Cc: xen-devel On Mon, Dec 01, Sander Eikelenboom wrote: > Monday, December 1, 2014, 3:34:09 PM, you wrote: > > actually be a workaround for the double pci-attach bug. > Don't know about that bug. You just replied to it. ;-) Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 13:32 ` Olaf Hering 2014-12-01 13:42 ` Olaf Hering @ 2014-12-01 17:01 ` Konrad Rzeszutek Wilk 2014-12-02 15:46 ` Olaf Hering 2014-12-04 1:31 ` Konrad Rzeszutek Wilk 1 sibling, 2 replies; 18+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-12-01 17:01 UTC (permalink / raw) To: Olaf Hering; +Cc: xen-devel On Mon, Dec 01, 2014 at 02:32:44PM +0100, Olaf Hering wrote: > On Mon, Dec 01, Olaf Hering wrote: > > > # xl pci-assignable-add 01:10.0 > > # xl pci-assignable-list > > 0000:01:10.0 > > # xl create -f domU.cfg > > # xl console domU > > ## lspci gives just emulated PCI devices > > ttyS0:Rescue:~ # lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff) > > > ## detach from domU console > > # xl pci-attach domU 0000:01:10.0 > > # xl pci-list domU > > Vdev Device > > 04.0 0000:01:10.0 > > > > # xl console domU > > ## lspci gives just emulated PCI devices > > ttyS0:Rescue:~ # lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01) > > > ## detach from domU console > > # xl pci-detach domU 0000:01:10.0 > Now lspci shows that the emulated network card is gone. > ttyS0:Rescue:~ # lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > > # xl pci-attach domU 0000:01:10.0 > > # xl pci-list domU > > Vdev Device > > 04.0 0000:01:10.0 > > # xl console domU > > ## lspci shows now also the assigned host device > > > So the actual bug is that the very first time after pci-attach the guests > "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just > the guest does not notice that "00:04.0" was actually already gone after unplug. That is odd - I see any device 'hot-plugged' being added at 00:05 and further. > > I wonder why the unplug code in xen_platform.c does just a qdev_free() instead > of a real pci-detach kind of thing. I'm sure this could be done at least for > emulated network cards. > > Olaf > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 17:01 ` Konrad Rzeszutek Wilk @ 2014-12-02 15:46 ` Olaf Hering 2014-12-02 18:39 ` Konrad Rzeszutek Wilk 2014-12-04 1:31 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 18+ messages in thread From: Olaf Hering @ 2014-12-02 15:46 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel On Mon, Dec 01, Konrad Rzeszutek Wilk wrote: > That is odd - I see any device 'hot-plugged' being added at 00:05 and further. Does this by any chance depend on the guest?! I mean, how is the guest notified that a PCI device is gone (by unplug)? Maybe the pvops case just happens to work because the unplug happens early, perhaps before PCI discovery?! Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-02 15:46 ` Olaf Hering @ 2014-12-02 18:39 ` Konrad Rzeszutek Wilk 2014-12-03 8:36 ` Olaf Hering 0 siblings, 1 reply; 18+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-12-02 18:39 UTC (permalink / raw) To: Olaf Hering; +Cc: xen-devel On Tue, Dec 02, 2014 at 04:46:52PM +0100, Olaf Hering wrote: > On Mon, Dec 01, Konrad Rzeszutek Wilk wrote: > > > That is odd - I see any device 'hot-plugged' being added at 00:05 and further. > > Does this by any chance depend on the guest?! I mean, how is the guest I doubt it. > notified that a PCI device is gone (by unplug)? Maybe the pvops case > just happens to work because the unplug happens early, perhaps before > PCI discovery?! ACPI hotplug. And it does work after PCI discovery. > > Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-02 18:39 ` Konrad Rzeszutek Wilk @ 2014-12-03 8:36 ` Olaf Hering 2014-12-03 11:24 ` Olaf Hering 0 siblings, 1 reply; 18+ messages in thread From: Olaf Hering @ 2014-12-03 8:36 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel On Tue, Dec 02, Konrad Rzeszutek Wilk wrote: > On Tue, Dec 02, 2014 at 04:46:52PM +0100, Olaf Hering wrote: > > On Mon, Dec 01, Konrad Rzeszutek Wilk wrote: > > > > > That is odd - I see any device 'hot-plugged' being added at 00:05 and further. > > > > Does this by any chance depend on the guest?! I mean, how is the guest > > I doubt it. Something is different here, likely just the non-pvops guest kernel. > > notified that a PCI device is gone (by unplug)? Maybe the pvops case > > just happens to work because the unplug happens early, perhaps before > > PCI discovery?! > > ACPI hotplug. And it does work after PCI discovery. In a pvops kernel, is the emulated but unplugged PCI hardware still listed with lspci? Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-03 8:36 ` Olaf Hering @ 2014-12-03 11:24 ` Olaf Hering 0 siblings, 0 replies; 18+ messages in thread From: Olaf Hering @ 2014-12-03 11:24 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel On Wed, Dec 03, Olaf Hering wrote: > On Tue, Dec 02, Konrad Rzeszutek Wilk wrote: > > On Tue, Dec 02, 2014 at 04:46:52PM +0100, Olaf Hering wrote: > > > On Mon, Dec 01, Konrad Rzeszutek Wilk wrote: > > ACPI hotplug. And it does work after PCI discovery. > In a pvops kernel, is the emulated but unplugged PCI hardware still listed with lspci? It is not. So thats why it happens to work. So how would I trigger an ACPI hotplug event within qemus unplug code? Olaf ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-01 17:01 ` Konrad Rzeszutek Wilk 2014-12-02 15:46 ` Olaf Hering @ 2014-12-04 1:31 ` Konrad Rzeszutek Wilk 2014-12-04 1:46 ` Konrad Rzeszutek Wilk 2014-12-04 9:28 ` Ian Campbell 1 sibling, 2 replies; 18+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-12-04 1:31 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: Olaf Hering, xen-devel On Mon, Dec 01, 2014 at 12:01:51PM -0500, Konrad Rzeszutek Wilk wrote: > On Mon, Dec 01, 2014 at 02:32:44PM +0100, Olaf Hering wrote: > > On Mon, Dec 01, Olaf Hering wrote: > > > > > # xl pci-assignable-add 01:10.0 > > > # xl pci-assignable-list > > > 0000:01:10.0 > > > # xl create -f domU.cfg > > > # xl console domU > > > ## lspci gives just emulated PCI devices > > > > ttyS0:Rescue:~ # lspci > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff) > > > > > ## detach from domU console > > > # xl pci-attach domU 0000:01:10.0 > > > # xl pci-list domU > > > Vdev Device > > > 04.0 0000:01:10.0 > > > > > > # xl console domU > > > ## lspci gives just emulated PCI devices > > > > ttyS0:Rescue:~ # lspci > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01) > > > > > ## detach from domU console > > > # xl pci-detach domU 0000:01:10.0 > > Now lspci shows that the emulated network card is gone. > > > ttyS0:Rescue:~ # lspci > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > > > > # xl pci-attach domU 0000:01:10.0 > > > # xl pci-list domU > > > Vdev Device > > > 04.0 0000:01:10.0 > > > # xl console domU > > > ## lspci shows now also the assigned host device > > > > > > So the actual bug is that the very first time after pci-attach the guests > > "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just > > the guest does not notice that "00:04.0" was actually already gone after unplug. > > That is odd - I see any device 'hot-plugged' being added at 00:05 and further. I have to apologize. The reason it works for me is because the emulated device gets unplugged quite early: # lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 [and here I run 'xl pci-attach USB 00:1a.0] # [ 30.609802] hpet1: lost 1589 rtc interrupts [ 30.672030] hpet1: lost 2200 rtc interrupts [ 30.672030] hpet1: lost 2201 rtc interrupts [ 30.672030] hpet1: lost 2200 rtc interrupts [ 30.899341] pci 0000:00:04.0: [8086:1c2d] type 00 class 0x0c0320 And the other test I have been running is when the guest is booted with an PCI device (and then unplugged): # lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) 00:05.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04) But oddly enough when I do 'pci-detach' and then 'pci-attach' I get: lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) 00:04.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04) so something is busted as you surmised And it looks to be busted in two cases - I see the 'hpet1' issues and the guest then crashes! > > > > > I wonder why the unplug code in xen_platform.c does just a qdev_free() instead > > of a real pci-detach kind of thing. I'm sure this could be done at least for > > emulated network cards. > > > > Olaf > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-04 1:31 ` Konrad Rzeszutek Wilk @ 2014-12-04 1:46 ` Konrad Rzeszutek Wilk 2014-12-04 9:28 ` Ian Campbell 1 sibling, 0 replies; 18+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-12-04 1:46 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: Olaf Hering, xen-devel On Wed, Dec 03, 2014 at 09:31:03PM -0400, Konrad Rzeszutek Wilk wrote: > On Mon, Dec 01, 2014 at 12:01:51PM -0500, Konrad Rzeszutek Wilk wrote: > > On Mon, Dec 01, 2014 at 02:32:44PM +0100, Olaf Hering wrote: > > > On Mon, Dec 01, Olaf Hering wrote: > > > > > > > # xl pci-assignable-add 01:10.0 > > > > # xl pci-assignable-list > > > > 0000:01:10.0 > > > > # xl create -f domU.cfg > > > > # xl console domU > > > > ## lspci gives just emulated PCI devices > > > > > > ttyS0:Rescue:~ # lspci > > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev ff) > > > > > > > ## detach from domU console > > > > # xl pci-attach domU 0000:01:10.0 > > > > # xl pci-list domU > > > > Vdev Device > > > > 04.0 0000:01:10.0 > > > > > > > > # xl console domU > > > > ## lspci gives just emulated PCI devices > > > > > > ttyS0:Rescue:~ # lspci > > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > > 00:04.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 01) > > > > > > > ## detach from domU console > > > > # xl pci-detach domU 0000:01:10.0 > > > Now lspci shows that the emulated network card is gone. > > > > > ttyS0:Rescue:~ # lspci > > > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > > > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > > > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > > > 00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) > > > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > > > > > > # xl pci-attach domU 0000:01:10.0 > > > > # xl pci-list domU > > > > Vdev Device > > > > 04.0 0000:01:10.0 > > > > # xl console domU > > > > ## lspci shows now also the assigned host device > > > > > > > > > So the actual bug is that the very first time after pci-attach the guests > > > "00:04.0" PCI device is (most likely) replaced with the host PCI device. Just > > > the guest does not notice that "00:04.0" was actually already gone after unplug. > > > > That is odd - I see any device 'hot-plugged' being added at 00:05 and further. > > I have to apologize. The reason it works for me is because the emulated > device gets unplugged quite early: > > # lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446 > > [and here I run 'xl pci-attach USB 00:1a.0] > > # [ 30.609802] hpet1: lost 1589 rtc interrupts > [ 30.672030] hpet1: lost 2200 rtc interrupts > [ 30.672030] hpet1: lost 2201 rtc interrupts > [ 30.672030] hpet1: lost 2200 rtc interrupts > [ 30.899341] pci 0000:00:04.0: [8086:1c2d] type 00 class 0x0c0320 > > And the other test I have been running is when the guest is booted > with an PCI device (and then unplugged): > > # lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) > 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 > 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) > 00:05.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04) > > But oddly enough when I do 'pci-detach' and then 'pci-attach' I get: > lspci > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) > 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 > 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01) > 00:04.0 USB Controller: Intel Corporation Cougar Point USB Enhanced Host Controller #2 (rev 04) > > so something is busted as you surmised > > And it looks to be busted in two cases - I see the 'hpet1' issues > and the guest then crashes! But that only shows up on this particular machine. If I use VFs and unplug/plug it works OK. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-04 1:31 ` Konrad Rzeszutek Wilk 2014-12-04 1:46 ` Konrad Rzeszutek Wilk @ 2014-12-04 9:28 ` Ian Campbell 2014-12-04 12:03 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 18+ messages in thread From: Ian Campbell @ 2014-12-04 9:28 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: Olaf Hering, xen-devel On Wed, 2014-12-03 at 21:31 -0400, Konrad Rzeszutek Wilk wrote: > so something is busted as you surmised I haven't been following closely, so this may be completely unrelated to the issue under discussion, but doesn't in-HVM-guest hotplug require a guest side driver to be loaded? I recall in the dim and distant past having to manually load some sort of foo_hp.ko. Ian. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: xl pci-attach silently fails the first time 2014-12-04 9:28 ` Ian Campbell @ 2014-12-04 12:03 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 18+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-12-04 12:03 UTC (permalink / raw) To: Ian Campbell; +Cc: Olaf Hering, xen-devel [-- Attachment #1.1: Type: text/plain, Size: 777 bytes --] On Dec 4, 2014 4:29 AM, "Ian Campbell" <Ian.Campbell@citrix.com> wrote: > > On Wed, 2014-12-03 at 21:31 -0400, Konrad Rzeszutek Wilk wrote: > > so something is busted as you surmised > > I haven't been following closely, so this may be completely unrelated to > the issue under discussion, but doesn't in-HVM-guest hotplug require a > guest side driver to be loaded? I recall in the dim and distant past > having to manually load some sort of foo_hp.ko. > The ACPI PCI hotplug machinery does all of that nicely. As mentioned - it works nicely - except that the BDF that is chosen is at an wrong slot. In the 'xend' days in recall this working nicely and sticking the PF at slot 5 and beyond. Though I have no clue how it dealt with multiple emulated NICs and such. > Ian. > [-- Attachment #1.2: Type: text/html, Size: 1015 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2014-12-04 12:03 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-12-01 12:57 xl pci-attach silently fails the first time Olaf Hering 2014-12-01 13:32 ` Olaf Hering 2014-12-01 13:42 ` Olaf Hering 2014-12-01 13:57 ` Sander Eikelenboom 2014-12-01 14:34 ` Olaf Hering 2014-12-01 14:39 ` Ian Campbell 2014-12-01 14:46 ` Olaf Hering 2014-12-01 14:46 ` Sander Eikelenboom 2014-12-02 15:47 ` Olaf Hering 2014-12-01 17:01 ` Konrad Rzeszutek Wilk 2014-12-02 15:46 ` Olaf Hering 2014-12-02 18:39 ` Konrad Rzeszutek Wilk 2014-12-03 8:36 ` Olaf Hering 2014-12-03 11:24 ` Olaf Hering 2014-12-04 1:31 ` Konrad Rzeszutek Wilk 2014-12-04 1:46 ` Konrad Rzeszutek Wilk 2014-12-04 9:28 ` Ian Campbell 2014-12-04 12:03 ` Konrad Rzeszutek Wilk
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.