From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wanpeng Li Subject: Re: [Question] About the behavior of HLT in VMX guest mode Date: Tue, 21 Mar 2017 14:21:55 +0800 Message-ID: References: <58C64672.1070706@huawei.com> <20170315173254.GF14081@potion> <58C9F3A5.3090604@huawei.com> <20170316142340.GD14076@potion> <58CB727B.20902@huawei.com> <20170320151815.GA25540@potion> <58D0863D.4080001@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , kvm , Gonglei , Paolo Bonzini To: "Longpeng (Mike)" Return-path: Received: from mail-wr0-f171.google.com ([209.85.128.171]:33586 "EHLO mail-wr0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753522AbdCUGWO (ORCPT ); Tue, 21 Mar 2017 02:22:14 -0400 Received: by mail-wr0-f171.google.com with SMTP id u48so105460262wrc.0 for ; Mon, 20 Mar 2017 23:21:56 -0700 (PDT) In-Reply-To: <58D0863D.4080001@huawei.com> Sender: kvm-owner@vger.kernel.org List-ID: 2017-03-21 9:47 GMT+08:00 Longpeng (Mike) : > Hi Radim, > > On 2017/3/20 23:18, Radim Kr=C4=8Dm=C3=A1=C5=99 wrote: > >> 2017-03-17 13:22+0800, Longpeng (Mike): >>> Hi Radim, > ... >>> In my humble opinion: >>> >>> 1) As "Intel sdm vol3 ch25.3" says, MWAIT operates normally (I think in= cludes >>> entering deeper sleep) under certain conditions. >>> Some deeper sleep modes(such as C4E/C6/C7) will clear the L1/L2/L3 cach= e. >>> This is insecurity if we don't take other protective measures(such as l= imit the >>> guest's max-cstate, it's fortunately that power subsystem isn't support= ed by >>> QEMU, but we should be careful for some special-purpose in case). While= HLT in >>> guest mode can't cause hardware into sleep. >> >> Good point. I'm not aware of any VMX capabilities to prevent deeper >> C-states, so we'd always hope that guests obey provided information. >> > > > I'll do some tests this weekend. > I plan to use MWAIT to enter deeper C-states in a testcase of kvm-unit-te= sts, > and start a memory-sensitive workload on another hyper-thread, then use > intel-pcm or perf to observe the count of cache miss on that core. > >>> 2) According to the "Intel sdm vol3 ch26.3.3 & ch27.5.6", I think MONIT= OR in >>> guest mode can't work as perfect as in host sometimes. >>> For example, a vcpu MONITOR a address and then MWAIT, if a external-int= r(suppose >>> this intr won't cause to inject any virtual events ) cause VMEXIT, the = monitor >>> address will be cleaned, so the MWAIT won't be waken up by a store oper= ation to >>> the monitored address any more. >> >> It's not as perfect, but should not cause a bug (well, there is a >> discussion with suspicious MWAIT behavior :]). >> MWAIT on all Intels I tested would just behave as a nop if exit happened >> between MONITOR and MWAIT, like it does if you skip the MONITOR (MWAIT >> instruction desciption): >> >> If the preceding MONITOR instruction did not successfully arm an >> address range or if the MONITOR instruction has not been executed >> prior to executing MWAIT, then the processor will not enter the >> implementation-dependent-optimized state. Execution will resume at the >> instruction following the MWAIT. >> > > > OK. :) > >>> But I'm glad to do some tests if time permits, thanks :) >>> >>> Radim, how about to make HLT-exiting configurable again in upstream ? I= f you >>> like it, there is a problem should be resolved, asynpf is conflict with >>> "HLT-exiting =3D 0" in certain situations. >> >> Go ahead. KVM should provide access to hardware features and >> no-HLT-exiting is reasonable as a per-VM (even per-VCPU if you make a >> good case) capability. I'm interested in the asyncpf conflict. After async pf setup successfully, there is a broadcast wakeup w/ special token 0xffffffff which tells vCPU that it should wake up all processes waiting for APFs though there is no real process waiting at the moment. Refer to SDM 26.3.1.5: HLT. The only events allowed are the following: =E2=80=94 Those with interruption type external interrupt or non-maskable interrupt (NMI). =E2=80=94 Those with interruption type hardware exception and vector 1 (deb= ug exception) or vector 18 (machine-check exception). =E2=80=94 Those with interruption type other event and vector 0 (pending MT= F VM exit). So if the guest activity state is hlt and delivery #PF event during vmentry, the vmentry will fail. Refer to the original "KVM: VMX: add module parameter to avoid trapping HLT instructions" https://www.spinics.net/lists/kvm-commits/msg00137.html, it will set guest activity state to active if it is hlt before manually. Actually I wonder who set guest activity state to active when there is HLT-exiting. In addition, what's your design for per-VM non HLT-exiting capability? > I had did some offline discussion with Wanpeng Li, he's interesting to wr= ite a > path for this feature. :) Thanks Longpeng. :) Regards, Wanpeng Li