From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [RFC 0/5] xen/arm: support big.little SoC Date: Tue, 20 Sep 2016 00:55:42 +0200 Message-ID: <1474325742.4393.78.camel@citrix.com> References: <1474250936-27962-1-git-send-email-peng.fan@nxp.com> <10152e13-bccb-0794-44e4-556845875e33@arm.com> <20160919083619.GA16854@linux-7smt.suse> <5ddefbc1-3bd4-c990-b615-0039761535d8@arm.com> <97d77bdb-2f4e-e89a-95b9-8aacb56eebc0@suse.com> <1474305482.4393.42.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2248729471446430047==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Stefano Stabellini Cc: Juergen Gross , Peng Fan , George Dunlap , Andrew Cooper , "xen-devel@lists.xen.org" , Julien Grall , Jan Beulich , Peng Fan List-Id: xen-devel@lists.xenproject.org --===============2248729471446430047== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-VZrQ9sqxVP9v4Pktn9da" --=-VZrQ9sqxVP9v4Pktn9da Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 2016-09-19 at 14:03 -0700, Stefano Stabellini wrote: > On Mon, 19 Sep 2016, Dario Faggioli wrote: > > Setting thing up like this, even automatically, either in > hypervisor or > > toolstack, is basically already possible (with all the good and bad > > aspects of pinning, of course). > >=C2=A0 > > Then, sure (as I said when replying to George), we may want things > to > > be more flexible, and we also probably want to be on the safe side > --if=C2=A0 > > ever some components manages to undo our automatic pinning-- wrt > the > > scheduler not picking up work for the wrong architecture... But > still > > I'm a bit surprised this did not came up... Julien, Peng, is that > > because you think this is not doable for any reason I'm missing? >=20 > Let's suppose that Xen detects big.LITTLE and pins dom0 vcpus to big > automatically. How can the user know that she really needs to be > careful > in the way she pins the vcpus of new VMs? Xen would also need to pin > automatically vcpus of new VMs to either big or LITTLE cores, or xl > would have to do it. >=20 Actually doing things with what we currently have for pinning is only something I've brought up as an example, and (potentially) useful for proof-of-concept, or very early stage level support. In the long run, when thinking to the scheduler based solution, I see things happening the other way round: you specify in xl config file (and with boot parameters, for dom0) how many big and how many LITTLE vcpus you want, and the scheduler will know that it can only schedule the big ones on big physical cores, and the LITTLE ones on LITTLE physical cores. Note that we're saying 'pinning' (yeah, I know, I did it myself in the first place :-/), but that would not be an actual 1-to-1 pinning. For instance, if domain X has 4 big pcpus, say 0,1,2,3,4, and the host has 8 big pcpus, say 8-15, then dXv1, dXv2, dXv3 and dXv4 will only be run by the scheduler on pcpus 8-15. Any of them, and with migration and load balancing within the set possible. This is what I'm talking about. And this would work even if/when there is only one cpupool, or in general for domains that are in a pool that has both big and LITTLE pcpus. Furthermore, big.LITTLE support and cpupools will be orthogonal, just like pinning and cpupools are orthogonal right now. I.e., once we will have what I described above, nothing prevents us from implementing per-vcpu cpupool membership, and either create the two (or more!) big and LITTLE pools, or from mixing things even more, for more complex and specific use cases. :-) Actually, with the cpupool solution, if you want a guest (or dom0) to actually have both big and LITTLE vcpus, you necessarily have to implement per-vcpu (rather than per-domain, as it is now) cpupool membership. I said myself it's not impossible, but certainly it's some work... with the scheduler solution you basically get that for free! So, basically, if we use cpupools for the basics of big.LITTLE support, there's no way out of it (apart from going implementing scheduling support afterwords, but that looks backwards to me, especially when thinking at it with the code in mind). > The whole process would be more explicit and obvious if we used > cpupools. It would be easier for users to know what it is going on -- > they just need to issue an `xl cpupool-list' command and they would > see > two clearly named pools (something like big-pool and LITTLE-pool).=20 > Well, I guess that, as part of big.LITTLE support, there will be a way to tell what pcpus are big and which are LITTLE anyway, probably both from `xl info' and from `xl cpupool-list -c' (and most likely in other ways too). > We > wouldn't have to pin vcpus to cpus automatically in Xen or xl, which > doesn't sound like fun. > As tried to say above, it will _look_ like some kind of automatic pinning, but that does not mean it has to be implemented by means of it, or dealt with by the user in the same way. Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-VZrQ9sqxVP9v4Pktn9da Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJX4GzvAAoJEBZCeImluHPudWsP/jEmnCzKPVjaxsJQYrrovNC2 iKqOM41toKVIAuF7KH+WUZZ0AbEdjRVpFXnrayIWLCkCgKpQYkoHDiTx9jVFtI8o wr2VdKXMhKQtnB12aUMbEmAGY0aT4B3BSTUPApDNLoeBBsry0hXy6rtsxEMYvpbc Oplg/NxbUWPCs1PiIoWBziMq/j+/y3pYr/bNLeBO1OKHzoZxKkh0t9IijPCZ52+Q mnmMgma8pH3I1oFbYcj5LnNsDBAJYmlB2RzEBD3ixsRnD0BAlS6w1Tp7Jws9wlyj JPoVam11u1wP8OfnUAihnOWZ97bzvr1AK8y5XkNrosRVzXt8BaV6IrLDpl/N2Fpi /rs1fiVUmOx9/ha1TPOO8IC0TAGXa+sfkKUuCbdMXK3hZ9xYr8uFgJtqWEIQIt0+ YY1wo/WCOZjfjuOPrsknBmYrE3Y77eM1G/IGmKdx0VP9Evox0oknHd0JDz0NZmQE uS0hBNiTJ2LCinE94RK1mJExkombQjAhg9wXniPxKJBFERZgnkNId3CIM5jQUvA6 4fwq5jzqsM1Lm7wu9CXU/40hRmCL2+z0vw+M4A1JvkOGFLKnjcD/Vf47AkWD0X27 l/inTzc82usXSrOGRbUeEqDMWJMaWEbhwz6GzMI/MHUXJBrdbvFfgjuoA/zYKcq0 DoJivB9c34LNQLJKkfiF =FYRk -----END PGP SIGNATURE----- --=-VZrQ9sqxVP9v4Pktn9da-- --===============2248729471446430047== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============2248729471446430047==--