From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [PATCH V3] libxl: Increase device model startup timeout to 1min. Date: Tue, 14 Jul 2015 12:52:29 +0200 Message-ID: <1436871149.13522.83.camel@citrix.com> References: <21915.58620.948343.728555@mariner.uk.xensource.com> <1436281753-19534-1-git-send-email-anthony.perard@citrix.com> <21915.60619.555732.214104@mariner.uk.xensource.com> <1436283671.25646.254.camel@citrix.com> <55A4C593020000780009077E@mail.emea.novell.com> <1436860520.7019.140.camel@citrix.com> <1436865941.13522.68.camel@citrix.com> <1436866654.25044.45.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4934524544450576563==" Return-path: In-Reply-To: <1436866654.25044.45.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell Cc: Wei Liu , Stefano Stabellini , Ian Jackson , xen-devel@lists.xen.org, Jan Beulich , Anthony PERARD List-Id: xen-devel@lists.xenproject.org --===============4934524544450576563== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-mjSkyocvydPWPYej+A7+" --=-mjSkyocvydPWPYej+A7+ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2015-07-14 at 10:37 +0100, Ian Campbell wrote: > On Tue, 2015-07-14 at 11:25 +0200, Dario Faggioli wrote: > > On Tue, 2015-07-14 at 08:55 +0100, Ian Campbell wrote: > > > It'll be hard to say until this change gets through the Xen push gate > > > and that version gets used for other branches (linux testing, libvirt= , > > > ovmf, osstest's own gate etc). > > >=20 > > Indeed. My opinion is that no, it is not. > >=20 > > My understanding of the data Anthony provided is that, under some > > (difficult to track/analyze/reproduce/etc) load conditions, the Linux I= O > > and VM subsystem suffer from high latency, delaying QEMU startup. > >=20 > > In the merlot* cases, the system is completely idle, apart from the > > failing creation/migration operation. > >=20 > > So, no, I don't think that would not be the fix we need for that > > situation. >=20 > Even if it is not the correct fix it seems like in some situations the > increase in timeout has improved things, hence it is an "answer" as Jan > asked (his quote marks). >=20 Sure! And that's why I find this weird/interesting... > > > At the moment it looks like it has helped with some but not all of th= e > > > issues. > > >=20 > > > These: > > >=20 > > > http://logs.test-lab.xenproject.org/osstest/results/host/merlot0.html > > > http://logs.test-lab.xenproject.org/osstest/results/host/merlot1.html > > >=20 > > Can I ask why (I mean, e.g., comparing what with what) you're saying it > > seems to have helped? >=20 > There seemed (unscientifically) to be fewer of the libvirt related > guest-start failures. >=20 And you mean by only looking at xen-unstable lines, don't you? If yes, looking at merlot0, I've found the below. Old timeout, failing: http://logs.test-lab.xenproject.org/osstest/logs/59105/test-amd64-amd64-lib= virt-xsm/info.html New timeout, success: http://logs.test-lab.xenproject.org/osstest/logs/59311/test-amd64-amd64-lib= virt/info.html And, looking at how long QEMU did take to start up that would be: 13:44:32 - 13:43:42 i.e., just a bit less than 1min! So, yes, it looks that this change is actually going to help in this case. What I'm missing is how it is possible that, on an idle system, DM spawning takes that long. As said, in Anthony's OpenStack case, the system was quite busy... not that it can't be a bug (somewhere, perhaps in Linux) in that case too, but here, it looks even more weird to me. May it be the NUMA misconfiguration? Well, if yes, I'm not sure how... Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-mjSkyocvydPWPYej+A7+ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlWk6fQACgkQk4XaBE3IOsSUUwCeM2I5ai0Qmkk0FvSXkB+ST2sW 72MAn2rVkv7Bdl84V7xxfsTpBZUE77BN =ZRK5 -----END PGP SIGNATURE----- --=-mjSkyocvydPWPYej+A7+-- --===============4934524544450576563== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============4934524544450576563==--