From: xudong <1921444@bugs.launchpad.net>
To: qemu-devel@nongnu.org
Subject: [Bug 1921444] Re: Q35 doesn't support to hot add the 2nd PCIe device to KVM guest
Date: Fri, 26 Mar 2021 05:10:29 -0000 [thread overview]
Message-ID: <161673542971.709.6194984416333720252.malone@soybean.canonical.com> (raw)
In-Reply-To: 161672454991.709.14499115006742419165.malonedeb@soybean.canonical.com
Alex, thanks for you response.
I tried to create guest with multiple root ports, but qemu report error:
# qemu-system-x86_64 --accel kvm -m 4096 -smp 4 -drive file=/home/hao/rhel8.2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0,mac=00:16:3e:0d:e4:ab -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper -cpu host -machine q35 -device pcie-root-port,id=root1 -device pcie-root-port,id=root2 -daemonize
qemu-system-x86_64: -device pcie-root-port,id=root2: Can't add chassis slot, error -16
Is the command wrong?
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1921444
Title:
Q35 doesn't support to hot add the 2nd PCIe device to KVM guest
Status in QEMU:
New
Bug description:
KVM: https://git.kernel.org/pub/scm/virt/kvm/kvm.git branch: next, commit: 4a98623d
Qemu: https://git.qemu.org/git/qemu.git branch: master, commit: 9e2e9fe3
Created a KVM guest with Q35 chipset, and try to hot add 2 PCIe device
to guest with qemu internal command device_add, the 1st device can be
added successfully, but the 2nd device failed to hot add.
If guest chipset is legacy i440fx, the 2 device can be added
successfully.
1. Enable VT-d in BIOS
2. load KVM modules in Linux OS: modprobe kvm; modprobe kvm_intel
3. Bind 2 device to vfio-pci
echo 0000:b1:00.0 > /sys/bus/pci/drivers/i40e/unbind
echo "8086 1572" > /sys/bus/pci/drivers/vfio-pci/new_id
echo 0000:b1:00.1 > /sys/bus/pci/drivers/i40e/unbind
echo "8086 1572" > /sys/bus/pci/drivers/vfio-pci/new_id
4. create guest with Q35 chipset:
qemu-system-x86_64 --accel kvm -m 4096 -smp 4 -drive file=/home/rhel8.2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -cpu host -machine q35 -device pcie-root-port,id=root1 -daemonize
5. hot add the 1st device to guest successfully
in guest qemu monitor "device_add vfio-pci,host=b1:00.0,id=nic0,bus=root1"
6. hot add the 2nd device to guest
in guest qemu monitor "device_add vfio-pci,host=b1:00.1,id=nic1,bus=root1"
The 2nd device doesn't be added in guest, and the 1st device is removed from guest.
Guest partial log:
[ 110.452272] pcieport 0000:00:04.0: pciehp: Slot(0): Attention button pressed
[ 110.453314] pcieport 0000:00:04.0: pciehp: Slot(0) Powering on due to button press
[ 110.454156] pcieport 0000:00:04.0: pciehp: Slot(0): Card present
[ 110.454792] pcieport 0000:00:04.0: pciehp: Slot(0): Link Up
[ 110.580927] pci 0000:01:00.0: [8086:1572] type 00 class 0x020000
[ 110.582560] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x007fffff 64bit pref]
[ 110.583453] pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00007fff 64bit pref]
[ 110.584278] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0007ffff pref]
[ 110.585051] pci 0000:01:00.0: Max Payload Size set to 128 (was 512, max 2048)
[ 110.586621] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 110.588140] pci 0000:01:00.0: BAR 0: no space for [mem size 0x00800000 64bit pref]
[ 110.588954] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x00800000 64bit pref]
[ 110.589797] pci 0000:01:00.0: BAR 6: assigned [mem 0xfe800000-0xfe87ffff pref]
[ 110.590703] pci 0000:01:00.0: BAR 3: assigned [mem 0xfe000000-0xfe007fff 64bit pref]
[ 110.592085] pcieport 0000:00:04.0: PCI bridge to [bus 01]
[ 110.592755] pcieport 0000:00:04.0: bridge window [io 0x1000-0x1fff]
[ 110.594403] pcieport 0000:00:04.0: bridge window [mem 0xfe800000-0xfe9fffff]
[ 110.595847] pcieport 0000:00:04.0: bridge window [mem 0xfe000000-0xfe1fffff 64bit pref]
[ 110.597867] PCI: No. 2 try to assign unassigned res
[ 110.597870] release child resource [mem 0xfe000000-0xfe007fff 64bit pref]
[ 110.597871] pcieport 0000:00:04.0: resource 15 [mem 0xfe000000-0xfe1fffff 64bit pref] released
[ 110.598881] pcieport 0000:00:04.0: PCI bridge to [bus 01]
[ 110.600789] pcieport 0000:00:04.0: BAR 15: assigned [mem 0x180000000-0x180bfffff 64bit pref]
[ 110.601731] pci 0000:01:00.0: BAR 0: assigned [mem 0x180000000-0x1807fffff 64bit pref]
[ 110.602849] pci 0000:01:00.0: BAR 3: assigned [mem 0x180800000-0x180807fff 64bit pref]
[ 110.604069] pcieport 0000:00:04.0: PCI bridge to [bus 01]
[ 110.604941] pcieport 0000:00:04.0: bridge window [io 0x1000-0x1fff]
[ 110.606237] pcieport 0000:00:04.0: bridge window [mem 0xfe800000-0xfe9fffff]
[ 110.607401] pcieport 0000:00:04.0: bridge window [mem 0x180000000-0x180bfffff 64bit pref]
[ 110.653661] i40e: Intel(R) Ethernet Connection XL710 Network Driver
[ 110.654443] i40e: Copyright (c) 2013 - 2019 Intel Corporation.
[ 110.655314] i40e 0000:01:00.0: enabling device (0140 -> 0142)
[ 110.672396] i40e 0000:01:00.0: fw 6.0.48442 api 1.7 nvm 6.01 0x800035b1 1.1747.0 [8086:1572] [8086:0008]
[ 110.750054] i40e 0000:01:00.0: MAC address: 3c:fd:fe:c0:59:98
[ 110.751792] i40e 0000:01:00.0: FW LLDP is enabled
[ 110.764644] i40e 0000:01:00.0 eth1: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[ 110.779390] i40e 0000:01:00.0: PCI-Express: Speed 8.0GT/s Width x8
[ 110.789841] i40e 0000:01:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 4 RSS FD_ATR FD_SB NTUPLE DCB VxLAN Geneve PTP VEPA
[ 111.817553] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[ 205.130288] pcieport 0000:00:04.0: pciehp: Slot(0): Attention button pressed
[ 205.131743] pcieport 0000:00:04.0: pciehp: Slot(0): Powering off due to button press
[ 205.133233] pcieport 0000:00:04.0: pciehp: Slot(0): Card not present
[ 205.135728] i40e 0000:01:00.0: i40e_ptp_stop: removed PHC on eth1
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1921444/+subscriptions
next prev parent reply other threads:[~2021-03-26 5:22 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-26 2:09 [Bug 1921444] [NEW] Q35 doesn't support to hot add the 2nd PCIe device to KVM guest xudong
2021-03-26 2:14 ` [Bug 1921444] " xudong
2021-03-26 4:01 ` Alex Williamson
2021-03-26 5:10 ` xudong [this message]
2021-05-14 19:03 ` Thomas Huth
2021-07-14 4:17 ` Launchpad Bug Tracker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=161673542971.709.6194984416333720252.malone@soybean.canonical.com \
--to=1921444@bugs.launchpad.net \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).