From: Ankur Arora <ankur.a.arora@oracle.com>
To: "Christoph Lameter (Ampere)" <cl@linux.com>
Cc: Mihai Carabas <mihai.carabas@oracle.com>,
linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org,
linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org,
catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de,
mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com,
pbonzini@redhat.com, wanpengli@tencent.com, vkuznets@redhat.com,
rafael@kernel.org, daniel.lezcano@linaro.org,
akpm@linux-foundation.org, pmladek@suse.com,
peterz@infradead.org, dianders@chromium.org, npiggin@gmail.com,
rick.p.edgecombe@intel.com, joao.m.martins@oracle.com,
juerg.haefliger@canonical.com, mic@digikod.net, arnd@arndb.de,
ankur.a.arora@oracle.com
Subject: Re: [PATCH 7/7] cpuidle/poll_state: replace cpu_relax with smp_cond_load_relaxed
Date: Thu, 30 Nov 2023 22:59:45 -0800 [thread overview]
Message-ID: <871qc6qufy.fsf@oracle.com> (raw)
In-Reply-To: <277fbd0d-25ea-437e-2ea7-6121c4e269db@linux.com>
Christoph Lameter (Ampere) <cl@linux.com> writes:
> On Wed, 22 Nov 2023, Mihai Carabas wrote:
>
>> La 22.11.2023 22:51, Christoph Lameter a scris:
>>> On Mon, 20 Nov 2023, Mihai Carabas wrote:
>>>
>>>> cpu_relax on ARM64 does a simple "yield". Thus we replace it with
>>>> smp_cond_load_relaxed which basically does a "wfe".
>>> Well it clears events first (which requires the first WFE) and then does a
>>> WFE waiting for any events if no events were pending.
>>> WFE does not cause a VMEXIT? Or does the inner loop of
>>> smp_cond_load_relaxed now do 2x VMEXITS?
>>> KVM ARM64 code seems to indicate that WFE causes a VMEXIT. See
>>> kvm_handle_wfx().
>>
>> In KVM ARM64 the WFE traping is dynamic: it is enabled only if there are more
>> tasks waiting on the same core (e.g. on an oversubscribed system).
>>
>> In arch/arm64/kvm/arm.c:
>>
>> 457 >-------if (single_task_running())
>> 458 >------->-------vcpu_clear_wfx_traps(vcpu);
>> 459 >-------else
>> 460 >------->-------vcpu_set_wfx_traps(vcpu);
>
> Ahh. Cool did not know about that. But still: Lots of VMEXITs once the load has
> to be shared.
Yeah, anytime there's more than one runnable process. Another, more
critical place where we will vmexit is the qspinlock slowpath which
uses smp_cond_load.
>> This of course can be improved by having a knob where you can completly
>> disable wfx traping by your needs, but I left this as another subject to
>> tackle.
Probably needs to be adaptive since we use WFE in error paths as well
(for instance to park the CPU.)
Ankur
next prev parent reply other threads:[~2023-12-01 7:04 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-20 14:01 [PATCH v2] Enable haltpoll for arm64 Mihai Carabas
2023-11-20 14:01 ` [PATCH 1/7] x86: Move ARCH_HAS_CPU_RELAX to arch Mihai Carabas
2023-11-27 14:44 ` Petr Mladek
2023-11-28 14:09 ` Mihai Carabas
2023-11-20 14:01 ` [PATCH 2/7] x86/kvm: Move haltpoll_want() to be arch defined Mihai Carabas
2023-11-29 20:55 ` Rafael J. Wysocki
2023-11-20 14:01 ` [PATCH 3/7] governors/haltpoll: Drop kvm_para_available() check Mihai Carabas
2023-11-20 14:01 ` [PATCH 4/7] arm64: Select ARCH_HAS_CPU_RELAX Mihai Carabas
2023-11-20 14:01 ` [PATCH 5/7] arm64: Define TIF_POLLING_NRFLAG Mihai Carabas
2023-11-20 14:01 ` [PATCH 6/7] cpuidle-haltpoll: ARM64 support Mihai Carabas
2023-11-20 14:01 ` [PATCH 7/7] cpuidle/poll_state: replace cpu_relax with smp_cond_load_relaxed Mihai Carabas
2023-11-22 20:51 ` Christoph Lameter
2023-11-22 21:33 ` Mihai Carabas
2023-11-27 20:17 ` Christoph Lameter (Ampere)
2023-12-01 6:59 ` Ankur Arora [this message]
2023-12-11 11:46 ` Will Deacon
2024-01-28 21:22 ` Mihai Carabas
2024-01-29 18:15 ` Will Deacon
2024-02-05 12:28 ` Mihai Carabas
2024-02-05 19:33 ` Ankur Arora
2024-01-17 21:19 ` [PATCH v2] Enable haltpoll for arm64 Christoph Lameter (Ampere)
2024-01-25 14:39 ` Mihai Carabas
2024-01-25 15:16 ` Rafael J. Wysocki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=871qc6qufy.fsf@oracle.com \
--to=ankur.a.arora@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=daniel.lezcano@linaro.org \
--cc=dianders@chromium.org \
--cc=hpa@zytor.com \
--cc=joao.m.martins@oracle.com \
--cc=juerg.haefliger@canonical.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mic@digikod.net \
--cc=mihai.carabas@oracle.com \
--cc=mingo@redhat.com \
--cc=npiggin@gmail.com \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=rafael@kernel.org \
--cc=rick.p.edgecombe@intel.com \
--cc=tglx@linutronix.de \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).