From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefano Stabellini Subject: Re: [PATCH] xen/arm: introduce vwfi parameter Date: Tue, 21 Feb 2017 09:49:21 -0800 (PST) Message-ID: References: <1487286292-29502-1-git-send-email-sstabellini@kernel.org> <484f2a39-1322-f40e-de01-97746e5de280@arm.com> <1487618436.6732.230.camel@citrix.com> <1845466e-c6a3-bcb9-0813-80ecdf267f03@arm.com> <1487631215.6732.266.camel@citrix.com> <06f20feb-0e9e-c757-b245-ebb55b422c67@arm.com> <1487668158.6732.310.camel@citrix.com> <14575011-0042-8940-c19f-2482136ff91c@foss.arm.com> <6a7b800a-dd6a-15a2-665d-d28352f24cd1@citrix.com> <1487689631.6732.412.camel@citrix.com> Mime-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="8323329-1928754916-1487699362=:12540" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cgEZ8-0003mh-Ko for xen-devel@lists.xenproject.org; Tue, 21 Feb 2017 17:49:30 +0000 In-Reply-To: <1487689631.6732.412.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Dario Faggioli Cc: edgar.iglesias@xilinx.com, Stefano Stabellini , george.dunlap@eu.citrix.com, Punit Agrawal , George Dunlap , Julien Grall , xen-devel@lists.xenproject.org, nd@arm.com List-Id: xen-devel@lists.xenproject.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-1928754916-1487699362=:12540 Content-Type: TEXT/PLAIN; charset=UTF-8 Content-Transfer-Encoding: 8BIT On Tue, 21 Feb 2017, Dario Faggioli wrote: > On Tue, 2017-02-21 at 13:46 +0000, George Dunlap wrote: > > > > A.  Don't trap guest WFI at all -- allow it to 'halt' in > > moderate-power-but-ready-for-interrupt mode. > > > > [..] > > > > A is safe because the scheduler should already have set a timer to > > break > > out of it if necessary.  The only potential issue here is that the > > guest > > is burning its credits, meaning that other vcpus with something > > potentially useful to do aren't being allowed to run; and then later > > when this vcpu has something useful to do it may be prevented from > > running because of low credits.  (This may be what Dario means when > > he > > says it "breaks scheduling"). > > > Are you also referring to the case when there are less vCPUs around > than the host has pCPUs (and, ideally, all vCPUs are pinned 1:1 to a > pCPU)? If yes, I agree that we're probably fine, but we have to check > and enforce all this to be the case. > > If no, think at a situation where there is 1 vCPU running on a pCPU and > 3 vCPUs in the runqueue (it may be a per-CPU Credit1 runqueue or a > shared among some pCPUs Credit2 runqueue). If the running vCPU goes > idle, let's say with WFI, we _don't_ want the pCPU to enter neither > moderate nor deep sleep, we want to pick up the first of the 3 other > vCPUs that are waiting in the runqueue. > > This is what I mean when I say "breaks scheduling". :-) That's right, I cannot see how this can be made to work correctly. > Oh, actually, if --which I only now realize may be what you are > referring to, since you're talking about "guest burning its credits"-- > you let the vCPU put the pCPU to sleep *but*, when it wakes up (or when > the scheduler runs again for whatever reason), you charge to it for all > the time the the pCPU was actually idle/sleeping, well, that may > actually  not break scheduling, or cause disruption to the service of > other vCPUs.... But indeed I'd consider it rather counter intuitive a > behavior. How can this be safe? There could be no interrupts programmed to wake up the pcpu at all. In fact, I don't think today there would be any, unless we set one up in Xen for the specific purpose of interrupting the pcpu sleep. I don't know the inner working of the scheduler, but does it always send an interrupt to other pcpu to schedule something? What if there are 2 vcpu pinned to the same pcpu? This cannot be fair. Obviously this behavior cannot be the default. But even if we introduce it as a command line option, when the user enables it can cause the whole system to hang if she is not careful. Isn't that right? We have no way to detect when we can do this safely. I prefer an option such as vwfi=pool, where in the worst case the user burns more power but there are no unexpected hangs. > In fact, it'd mean that the guest has issued WFI because he wanted to > sleep and we do put it to sleep. But when it wakes up, we treat it like > it had busy waited. > > What would be the benefit of this? That we don't context switch (either > to idle or to someone else)? Yes, that would be the benefit. --8323329-1928754916-1487699362=:12540 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --8323329-1928754916-1487699362=:12540--