From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: [RFC 0/5] xen/arm: support big.little SoC Date: Tue, 20 Sep 2016 02:54:06 +0200 Message-ID: <1474332846.4393.153.camel@citrix.com> References: <1474250936-27962-1-git-send-email-peng.fan@nxp.com> <10152e13-bccb-0794-44e4-556845875e33@arm.com> <20160919083619.GA16854@linux-7smt.suse> <5ddefbc1-3bd4-c990-b615-0039761535d8@arm.com> <97d77bdb-2f4e-e89a-95b9-8aacb56eebc0@suse.com> <1474305482.4393.42.camel@citrix.com> <1474325742.4393.78.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6550259976819462681==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Stefano Stabellini Cc: Juergen Gross , Peng Fan , George Dunlap , Andrew Cooper , "xen-devel@lists.xen.org" , Julien Grall , Jan Beulich , Peng Fan List-Id: xen-devel@lists.xenproject.org --===============6550259976819462681== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-BBmaWR/HogHQWJnGtL1V" --=-BBmaWR/HogHQWJnGtL1V Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, 2016-09-19 at 17:01 -0700, Stefano Stabellini wrote: > On Tue, 20 Sep 2016, Dario Faggioli wrote: > > And this would work even if/when there is only one cpupool, or in > > general for domains that are in a pool that has both big and LITTLE > > pcpus. Furthermore, big.LITTLE support and cpupools will be > > orthogonal, > > just like pinning and cpupools are orthogonal right now. I.e., once > > we > > will have what I described above, nothing prevents us from > > implementing > > per-vcpu cpupool membership, and either create the two (or more!) > > big > > and LITTLE pools, or from mixing things even more, for more complex > > and > > specific use cases. :-) >=20 > I think that everybody agrees that this is the best long term > solution. >=20 Well, no, that wasn't obvious to me. If that's the case, it's already something! :-) > >=20 > > Actually, with the cpupool solution, if you want a guest (or dom0) > > to > > actually have both big and LITTLE vcpus, you necessarily have to > > implement per-vcpu (rather than per-domain, as it is now) cpupool > > membership. I said myself it's not impossible, but certainly it's > > some > > work... with the scheduler solution you basically get that for > > free! > >=20 > > So, basically, if we use cpupools for the basics of big.LITTLE > > support, > > there's no way out of it (apart from going implementing scheduling > > support afterwords, but that looks backwards to me, especially when > > thinking at it with the code in mind). >=20 > The question is: what is the best short-term solution we can ask Peng > to > implement that allows Xen to run on big.LITTLE systems today? > Possibly > getting us closer to the long term solution, or at least not farther > from it? >=20 So, I still have to look closely at the patches in these series. But, with Credit2 in mind, if one: =C2=A0- take advantage of the knowledge of what arch a pcpu belongs inside= =C2=A0 =C2=A0 =C2=A0the code that arrange the pcpus in runqueues, which means we'l= l end=C2=A0 =C2=A0 =C2=A0up with big runqueues and LITTLE runqueues. I re-wrote that co= de, I =C2=A0 =C2=A0can provide pointers and help, if necessary; =C2=A0- tweak the one or two instance of for_each_runqueue() [*] that there =C2=A0 =C2=A0are in the code into a for_each_runqueue_of_same_class(), i.e.= : =C2=A0if (is_big(this_cpu)) =C2=A0{ =C2=A0 =C2=A0for_each_big_runqueue() =C2=A0 =C2=A0{ =C2=A0 =C2=A0 =C2=A0 .. =C2=A0 =C2=A0} =C2=A0} =C2=A0else =C2=A0{ =C2=A0 =C2=A0for_each_LITTLE_runqueue() =C2=A0 =C2=A0{ =C2=A0 =C2=A0 =C2=A0.. =C2=A0 =C2=A0} =C2=A0}=C2=A0 then big.LITTLE support in Credit2 would be done already, and all it would be left is support for the syntax of new config switches in xl, and a way of telling, from xl/libxl down to Xen, what arch a vcpu belongs to, so that it can be associated with one runqueue of the proper class. Thinking to Credit1, we need to make sure thet, in load_balance() and runq_steal(), a LITTLE cpu *only* ever try to steal work from another LITTLE cpu, and __never__ from a big cpu (and vice versa). And also that when a vcpu wakes up, and what it has in its v->processor is a LITTLE pcpu, that only LITTLE processors are considered for being tickled (I'm less certain of this last part, but it should be more or less like this). Then, of course the the same glue and vcpu classification code. However, in Credit1, it's possible that a trick like that would affect the accounting and credit algorithm, and hence provide unfair, or in general, unexpected results. Credit2 should, OTOH, be a lot mere resilient, wrt that. > > > The whole process would be more explicit and obvious if we used > > > cpupools. It would be easier for users to know what it is going > > > on -- > > > they just need to issue an `xl cpupool-list' command and they > > > would > > > see > > > two clearly named pools (something like big-pool and LITTLE- > > > pool).=C2=A0 > > >=20 > > Well, I guess that, as part of big.LITTLE support, there will be a > > way > > to tell what pcpus are big and which are LITTLE anyway, probably > > both > > from `xl info' and from `xl cpupool-list -c' (and most likely in > > other > > ways too). >=20 > Sure, but it needs to be very clear. We cannot ask people to spot > architecture specific flags among the output of `xl info' to be able > to > appropriately start a guest.=20 > As mentioned in previous mail, and as drafted when replying to Peng, the only think that the user should know is how many big and how many LITTLE vcpus she wants (and, potentially, which one would be each). :-) Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-BBmaWR/HogHQWJnGtL1V Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJX4IivAAoJEBZCeImluHPuRL0QANYxbPW1hwO8HOByuHOr0dkM FS6f+rBcyoma9OFFa0dc0Kl2DDIBzcWpJ/QLerf5nz/NWM7434ljTdrfqKyQau2C d0iATyb6b6J8dV6jwPyIvEW+2zY9pIozAxCGZEyNNyUlPkfd3qhJ0E7JII3EnXxh yxJdyluD1oo03R03uV+6SKCTGPodlnniHbuJ++kYus05annBuEkaUQ41YTc60Znq nVxvbMvMrGUy1ceIdvnKgg48TZIpp3l0guTPo3jjntmWqB0oVf0Ev0fIOWDn3Sr0 i9RXi31fmPWalp4wwrIIOGJKb4axhKO9z8pfP7xXKebDlLtU5ashq1uHtmVyA6GK JPj2TgIjt3pDOevATjYWXXtPFpA6DI6TD9rvp2N6ahnwOGRbHsEy5hNbSC0b7+Qm GpxKkgpnqnuFjR5v503TEWkorJXKZfHmjjhD1B/fQ7MMSJ/wYwDYKjJ1WyF8FPfY KeT8aM4xQiXSHG9MfsUKfXHeohsNyZZ9KwCsQfCSraQjZpkmsY67y4YCuvSMbNQa WP29+N78fVtulL7pXeh7hna8RmZ4wy5N11XJrHfWjBj3q2/d8aaRjZUB+jSXnSu8 ILGTJNViGuIL9GI5BsQeUeZg33ePWM/Z7cyI9wzfx19HXI1Epfwia0ZvtcWdQq2j VkSUF6LbcoMUJUoJN6lO =j/ef -----END PGP SIGNATURE----- --=-BBmaWR/HogHQWJnGtL1V-- --===============6550259976819462681== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============6550259976819462681==--