From: David Woodhouse <dwmw2@infradead.org>
To: Thomas Gleixner <tglx@linutronix.de>,
Usama Arif <usama.arif@bytedance.com>,
kim.phillips@amd.com
Cc: arjan@linux.intel.com, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, hpa@zytor.com, x86@kernel.org,
pbonzini@redhat.com, paulmck@kernel.org,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com,
thomas.lendacky@amd.com, seanjc@google.com,
pmenzel@molgen.mpg.de, fam.zheng@bytedance.com,
punit.agrawal@bytedance.com, simon.evans@bytedance.com,
liangma@liangbit.com, Ashok Raj <ashok.raj@intel.com>
Subject: Re: [PATCH v9 0/8] Parallel CPU bringup for x86_64
Date: Thu, 23 Feb 2023 15:12:00 +0000 [thread overview]
Message-ID: <eb6717dfc4ceb99803c0396f950db7c3231c75ef.camel@infradead.org> (raw)
In-Reply-To: <87zg94la7l.ffs@tglx>
[-- Attachment #1: Type: text/plain, Size: 2828 bytes --]
On Thu, 2023-02-23 at 15:37 +0100, Thomas Gleixner wrote:
> David!
>
> On Thu, Feb 23 2023 at 11:07, David Woodhouse wrote:
> > On Wed, 2023-02-22 at 17:42 +0100, Thomas Gleixner wrote:
> > > The low hanging fruit which brings most is the identification/topology
> > > muck and the microcode loading. That needs to be addressed first anyway.
> >
> > Agreed, thanks.
>
> So the problem with microcode loading is that we must ensure that a HT
> sibling is not executing anything else than a trivial loop waiting for
> the update to complete. So something like this should work:
>
> 1) Kick all CPUs into life and let them run up to cpu_init() and
> retrieve only the topology information.
>
> 2) Wait for all CPUs to reach this point
>
> 3) Release all primary HT threads so they can load microcode in
> parallel. The secondary HT threads stay in the wait loop and are
> released once the primary thread has finished the microcode
> update.
>
> 4) Let the CPUs do the full CPUID readout and let them synchronize
> with the control CPU again.
>
> 5) Complete bringup one by one
Can we move the microcode loading to happen earlier, during the x86-
specific CPUHP_BP_PARALLEL_DYN stage(s) while they're running in
parallel.
In the existing set of patches, we send INIT/SIPI/SIPI to each CPU in
parallel and they run to the first part of start_secondary(), up to the
point where it calls cpu_init_secondary() and sets their bit in
cpu_initialized_mask, then spinning and waiting for cpu_callout_mask.
My "part 2" test patch does another round in parallel, setting each
CPU's bit in 'cpu_callout_mask' and letting them run a bit further
through start_secondary() until they get to the end of smp_callin(),
where they set their bit in smp_callin_mask and (in my patch) wait for
their bit in a new cpu_finishup_mask to be set — which is what releases
them to proceed to completion in the final native_cpu_up() bringup.
So perhaps the BSP doesn't need to coordinate anything here, if we can
let the siblings work it out between themselves in the (now-)parallel
stage at the end of smp_callin()? And only set their bit in
smp_callin_mask when the microcode update is done?
Hm, maybe it's as simple as the first¹ thread on a core waiting for all
its siblings' bits in cpu_callin_mask to be set, and *then* doing the
update before setting its own bit?
¹ As long as we define "first" as the one with the lowest CPU#, which
means that the BSP won't release any of the siblings before it releases
the "first".
Then the siblings are just spinning on cpu_callin_mask anyway; they
don't need to do anything *more*.
Probably worth knocking it up and seeing how badly it explodes?
[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]
next prev parent reply other threads:[~2023-02-23 15:12 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-15 14:54 [PATCH v9 0/8] Parallel CPU bringup for x86_64 Usama Arif
2023-02-15 14:54 ` [PATCH v9 1/8] x86/apic/x2apic: Allow CPU cluster_mask to be populated in parallel Usama Arif
2023-02-16 20:58 ` Kim Phillips
2023-02-15 14:54 ` [PATCH v9 2/8] cpu/hotplug: Move idle_thread_get() to <linux/smpboot.h> Usama Arif
2023-02-15 14:54 ` [PATCH v9 3/8] cpu/hotplug: Add dynamic parallel bringup states before CPUHP_BRINGUP_CPU Usama Arif
2023-02-15 14:54 ` [PATCH v9 4/8] x86/smpboot: Reference count on smpboot_setup_warm_reset_vector() Usama Arif
2023-02-15 14:54 ` [PATCH v9 5/8] x86/smpboot: Split up native_cpu_up into separate phases and document them Usama Arif
2023-02-15 14:54 ` [PATCH v9 6/8] x86/smpboot: Support parallel startup of secondary CPUs Usama Arif
2023-02-15 14:54 ` [PATCH v9 7/8] x86/smpboot: Send INIT/SIPI/SIPI to secondary CPUs in parallel Usama Arif
2023-02-15 14:54 ` [PATCH v9 8/8] x86/smpboot: Serialize topology updates for secondary bringup Usama Arif
2023-02-16 6:34 ` [PATCH v9 0/8] Parallel CPU bringup for x86_64 Paul E. McKenney
2023-02-20 16:08 ` Oleksandr Natalenko
2023-02-20 16:20 ` David Woodhouse
2023-02-20 16:40 ` Oleksandr Natalenko
2023-02-20 20:31 ` David Woodhouse
2023-02-20 21:23 ` Oleksandr Natalenko
2023-02-20 21:34 ` Piotr Gorski
2023-02-20 21:39 ` David Woodhouse
2023-02-20 23:23 ` Kim Phillips
2023-02-20 23:30 ` David Woodhouse
2023-02-21 4:20 ` Kim Phillips
2023-02-21 7:16 ` David Woodhouse
2023-02-21 7:27 ` Oleksandr Natalenko
2023-02-21 7:53 ` David Woodhouse
2023-02-21 8:05 ` Oleksandr Natalenko
2023-02-21 8:17 ` David Woodhouse
2023-02-21 8:25 ` Oleksandr Natalenko
2023-02-21 8:35 ` David Woodhouse
2023-02-21 8:44 ` Oleksandr Natalenko
2023-02-21 9:06 ` David Woodhouse
2023-02-21 9:49 ` Oleksandr Natalenko
2023-02-21 10:27 ` David Woodhouse
2023-02-21 10:47 ` [External] " Usama Arif
2023-02-21 11:42 ` Oleksandr Natalenko
2023-02-21 11:54 ` Usama Arif
2023-02-21 13:22 ` David Woodhouse
2023-02-21 11:46 ` Oleksandr Natalenko
2023-02-21 11:49 ` David Woodhouse
2023-02-21 12:14 ` Oleksandr Natalenko
2023-02-21 19:10 ` David Woodhouse
2023-02-21 20:04 ` [External] " Usama Arif
2023-02-21 21:04 ` Oleksandr Natalenko
2023-02-21 21:41 ` Thomas Gleixner
2023-02-21 21:44 ` David Woodhouse
2023-02-21 23:18 ` David Woodhouse
2023-02-22 0:00 ` [External] " Usama Arif
2023-02-22 8:19 ` David Woodhouse
2023-02-22 9:46 ` Thomas Gleixner
2023-02-22 9:51 ` David Woodhouse
2023-02-22 9:31 ` Thomas Gleixner
2023-02-20 22:22 ` Piotr Gorski
2023-02-20 22:23 ` [External] " Usama Arif
2023-02-20 22:41 ` Oleksandr Natalenko
2023-02-22 10:11 ` David Woodhouse
2023-02-22 11:11 ` [External] " Usama Arif
2023-02-22 12:08 ` Brian Gerst
2023-02-22 12:53 ` David Woodhouse
2023-02-22 16:42 ` Thomas Gleixner
2023-02-23 11:07 ` David Woodhouse
2023-02-23 14:37 ` Thomas Gleixner
2023-02-23 15:12 ` David Woodhouse [this message]
2023-02-23 19:24 ` [External] " Usama Arif
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=eb6717dfc4ceb99803c0396f950db7c3231c75ef.camel@infradead.org \
--to=dwmw2@infradead.org \
--cc=arjan@linux.intel.com \
--cc=ashok.raj@intel.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=fam.zheng@bytedance.com \
--cc=hewenliang4@huawei.com \
--cc=hpa@zytor.com \
--cc=kim.phillips@amd.com \
--cc=kvm@vger.kernel.org \
--cc=liangma@liangbit.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mimoja@mimoja.de \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=pbonzini@redhat.com \
--cc=pmenzel@molgen.mpg.de \
--cc=punit.agrawal@bytedance.com \
--cc=rcu@vger.kernel.org \
--cc=seanjc@google.com \
--cc=simon.evans@bytedance.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=usama.arif@bytedance.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).