From: Sander Eikelenboom <linux@eikelenboom.it>
To: Anthony PERARD <anthony.perard@citrix.com>,
xen-devel@lists.xenproject.org, Chao Gao <chao.gao@intel.com>
Cc: Wei Liu <wl@xen.org>
Subject: Re: [Xen-devel] [PATCH 2/2] libxl_pci: Fix guest shutdown with PCI PT attached
Date: Thu, 10 Oct 2019 18:13:43 +0200 [thread overview]
Message-ID: <d12ee001-7f8e-4482-2a78-9cb1fd2d7530@eikelenboom.it> (raw)
In-Reply-To: <20191001103522.GG1163@perard.uk.xensource.com>
On 01/10/2019 12:35, Anthony PERARD wrote:
> Rewrite of the commit message:
>
> Before the problematic commit, libxl used to ignore error when
> destroying (force == true) a passthrough device, especially error that
> happens when dealing with the DM.
>
> Since fae4880c45fe, if the DM failed to detach the pci device within
> the allowed time, the timed out error raised skip part of
> pci_remove_*, but also raise the error up to the caller of
> libxl__device_pci_destroy_all, libxl__destroy_domid, and thus the
> destruction of the domain fails.
>
> In this patch, if the DM didn't confirmed that the device is removed,
> we will print a warning and keep going if force=true. The patch
> reorder the functions so that pci_remove_timeout() calls
> pci_remove_detatched() like it's done when DM calls are successful.
>
> We also clean the QMP states and associated timeouts earlier, as soon
> as they are not needed anymore.
>
> Reported-by: Sander Eikelenboom <linux@eikelenboom.it>
> Fixes: fae4880c45fe015e567afa223f78bf17a6d98e1b
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>
Hi Anthony / Chao,
I have to come back to this, a bit because perhaps there is an underlying issue.
While it earlier occurred to me that the VM to which I passed through most pci-devices
(8 to be exact) became very slow to shutdown, but I didn't investigate it further.
But after you commit messages from this patch it kept nagging, so today I did some testing
and bisecting.
The difference in tear-down time at least from what the IOMMU code logs is quite large:
xen-4.12.0
Setup: 7.452 s
Tear-down: 7.626 s
xen-unstable-ee7170822f1fc209f33feb47b268bab35541351d
Setup: 7.468 s
Tear-down: 50.239 s
Bisection turned up:
commit c4b1ef0f89aa6a74faa4618ce3efed1de246ec40
Author: Chao Gao <chao.gao@intel.com>
Date: Fri Jul 19 10:24:08 2019 +0100
libxl_qmp: wait for completion of device removal
Which makes me wonder if there is something going wrong in Qemu ?
--
Sander
xen-4.12.0 setup:
(XEN) [2019-10-10 09:54:14.846] AMD-Vi: Disable: device id = 0x900, domain = 0, paging mode = 3
(XEN) [2019-10-10 09:54:14.846] AMD-Vi: Setup I/O page table: device id = 0x900, type = 0x1, root table = 0x4aa847000, domain = 1, paging mode = 3
(XEN) [2019-10-10 09:54:14.846] AMD-Vi: Re-assign 0000:09:00.0 from dom0 to dom1
...
(XEN) [2019-10-10 09:54:22.298] AMD-Vi: Disable: device id = 0x907, domain = 0, paging mode = 3
(XEN) [2019-10-10 09:54:22.298] AMD-Vi: Setup I/O page table: device id = 0x907, type = 0x1, root table = 0x4aa847000, domain = 1, paging mode = 3
(XEN) [2019-10-10 09:54:22.298] AMD-Vi: Re-assign 0000:09:00.7 from dom0 to dom1
xen-4.12.0 tear-down:
(XEN) [2019-10-10 10:01:11.971] AMD-Vi: Disable: device id = 0x900, domain = 1, paging mode = 3
(XEN) [2019-10-10 10:01:11.971] AMD-Vi: Setup I/O page table: device id = 0x900, type = 0x1, root table = 0x53572c000, domain = 0, paging mode = 3
(XEN) [2019-10-10 10:01:11.971] AMD-Vi: Re-assign 0000:09:00.0 from dom1 to dom0
...
(XEN) [2019-10-10 10:01:19.597] AMD-Vi: Disable: device id = 0x907, domain = 1, paging mode = 3
(XEN) [2019-10-10 10:01:19.597] AMD-Vi: Setup I/O page table: device id = 0x907, type = 0x1, root table = 0x53572c000, domain = 0, paging mode = 3
(XEN) [2019-10-10 10:01:19.597] AMD-Vi: Re-assign 0000:09:00.7 from dom1 to dom0
xen-unstable-ee7170822f1fc209f33feb47b268bab35541351d setup:
(XEN) [2019-10-10 10:21:38.549] d1: bind: m_gsi=47 g_gsi=36 dev=00.00.5 intx=0
(XEN) [2019-10-10 10:21:38.621] AMD-Vi: Disable: device id = 0x900, domain = 0, paging mode = 3
(XEN) [2019-10-10 10:21:38.621] AMD-Vi: Setup I/O page table: device id = 0x900, type = 0x1, root table = 0x4aa83b000, domain = 1, paging mode = 3
(XEN) [2019-10-10 10:21:38.621] AMD-Vi: Re-assign 0000:09:00.0 from dom0 to dom1
...
(XEN) [2019-10-10 10:21:46.069] d1: bind: m_gsi=46 g_gsi=36 dev=00.01.4 intx=3
(XEN) [2019-10-10 10:21:46.089] AMD-Vi: Disable: device id = 0x907, domain = 0, paging mode = 3
(XEN) [2019-10-10 10:21:46.089] AMD-Vi: Setup I/O page table: device id = 0x907, type = 0x1, root table = 0x4aa83b000, domain = 1, paging mode = 3
(XEN) [2019-10-10 10:21:46.089] AMD-Vi: Re-assign 0000:09:00.7 from dom0 to dom1
xen-unstable-ee7170822f1fc209f33feb47b268bab35541351d tear-down:
(XEN) [2019-10-10 10:23:53.167] AMD-Vi: Disable: device id = 0x900, domain = 1, paging mode = 3
(XEN) [2019-10-10 10:23:53.167] AMD-Vi: Setup I/O page table: device id = 0x900, type = 0x1, root table = 0x5240f8000, domain = 0, paging mode = 3
(XEN) [2019-10-10 10:23:53.167] AMD-Vi: Re-assign 0000:09:00.0 from dom1 to dom0
...
(XEN) [2019-10-10 10:24:43.406] AMD-Vi: Disable: device id = 0x907, domain = 1, paging mode = 3
(XEN) [2019-10-10 10:24:43.406] AMD-Vi: Setup I/O page table: device id = 0x907, type = 0x1, root table = 0x5240f8000, domain = 0, paging mode = 3
(XEN) [2019-10-10 10:24:43.406] AMD-Vi: Re-assign 0000:09:00.7 from dom1 to dom0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-10-10 16:13 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-30 17:23 [Xen-devel] [PATCH 0/2] libxl fixes with pci passthrough Anthony PERARD
2019-09-30 17:23 ` [Xen-devel] [PATCH 1/2] libxl_pci: Don't ignore PCI PT error at guest creation Anthony PERARD
2019-09-30 17:23 ` [Xen-devel] [PATCH 2/2] libxl_pci: Fix guest shutdown with PCI PT attached Anthony PERARD
2019-09-30 18:10 ` Sander Eikelenboom
2019-10-01 10:35 ` Anthony PERARD
2019-10-10 16:13 ` Sander Eikelenboom [this message]
2019-10-14 15:03 ` Chao Gao
2019-10-15 16:59 ` Sander Eikelenboom
2019-10-15 18:46 ` Sander Eikelenboom
2019-10-16 4:55 ` Chao Gao
2019-10-18 16:11 ` Anthony PERARD
2019-10-18 16:43 ` Sander Eikelenboom
2019-10-02 15:45 ` [Xen-devel] [PATCH 0/2] libxl fixes with pci passthrough Ian Jackson
2019-10-02 15:58 ` Jürgen Groß
2019-10-04 15:55 ` Ian Jackson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d12ee001-7f8e-4482-2a78-9cb1fd2d7530@eikelenboom.it \
--to=linux@eikelenboom.it \
--cc=anthony.perard@citrix.com \
--cc=chao.gao@intel.com \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).