From: David Woodhouse <dwmw2@infradead.org>
To: Thomas Gleixner <tglx@linutronix.de>,
Paul McKenney <paulmckrcu@gmail.com>
Cc: Andy Lutomirski <luto@kernel.org>,
"shenkai (D)" <shenkai8@huawei.com>,
"Schander, Johanna 'Mimoja' Amelie" <mimoja@amazon.com>,
LKML <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
X86 ML <x86@kernel.org>, "H. Peter Anvin" <hpa@zytor.com>,
hewenliang4@huawei.com, hushiyuan@huawei.com,
luolongjun@huawei.com, hejingxian <hejingxian@huawei.com>
Subject: Re: [PATCH] use x86 cpu park to speedup smp_init in kexec situation
Date: Wed, 08 Dec 2021 14:14:35 +0000 [thread overview]
Message-ID: <d7939a95731de8b8eb9245c330f014772e40f145.camel@infradead.org> (raw)
In-Reply-To: <5039f6178715dc4725a8c7f071dfd9ef5d70ae43.camel@infradead.org>
[-- Attachment #1: Type: text/plain, Size: 3248 bytes --]
+Paul for the RCU question.
On Tue, 2021-02-16 at 15:10 +0000, David Woodhouse wrote:
> On Tue, 2021-02-16 at 13:53 +0000, David Woodhouse wrote:
> > I threw it into my tree at
> > https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/parallel
> >
> > It seems to work fairly nicely. The parallel SIPI seems to win be about
> > a third of the bringup time on my 28-thread Haswell box. This is at the
> > penultimate commit of the above branch:
> >
> > [ 0.307590] smp: Bringing up secondary CPUs ...
> > [ 0.307826] x86: Booting SMP configuration:
> > [ 0.307830] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14
> > [ 0.376677] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.
> > [ 0.377177] #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27
> > [ 0.402323] Brought CPUs online in 246691584 cycles
> > [ 0.402323] smp: Brought up 1 node, 28 CPUs
> >
> > ... and this is the tip of the branch:
> >
> > [ 0.308332] smp: Bringing up secondary CPUs ...<dwmw2_gone>
> > [ 0.308569] x86: Booting SMP configuration:
> > [ 0.308572] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27
> > [ 0.321120] Brought 28 CPUs to x86/cpu:kick in 34828752 cycles
> > [ 0.366663] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.
> > [ 0.368749] Brought CPUs online in 124913032 cycles
> > [ 0.368749] smp: Brought up 1 node, 28 CPUs
> > [ 0.368749] smpboot: Max logical packages: 1
> > [ 0.368749] smpboot: Total of 28 processors activated (145259.85 BogoMIPS)
> >
> > There's more to be gained here if we can fix up the next stage. Right
> > now if I set every CPU's bit in cpu_initialized_mask to allow them to
> > proceed from wait_for_master_cpu() through to the end of cpu_init() and
> > onwards through start_secondary(), they all end up hitting
> > check_tsc_sync_target() in parallel and it goes horridly wrong.
>
> Actually it breaks before that, in rcu_cpu_starting(). A spinlock
> around that, an atomic_t to let the APs do their TSC sync one at a time
> (both in the above tree now), and I have a 75% saving on CPU bringup
> time for my 28-thread Haswell:
Coming back to this, I've updated it and thrown up a new branch at
https://git.infradead.org/users/dwmw2/linux.git/shortlog/refs/heads/parallel-5.16
For those last two fixes I had started with a trivial naïve approach of
just enforcing serialization.
I'm sure we can come up with a cleverer 1:N way of synchronizing the
TSCs, instead of just serializing the existing 1:1 sync.
For rcu_cpu_starting() I see there's *already* a lock in the
rcu_node... could we use that same lock to protect the manipulation of
rnp->ofl_seq and allow rcu_cpu_starting() to be invoked by multiple APs
in parallel? Paul?
On a related note, are you currently guaranteed that rcu_report_dead()
cannot be called more than once in parallel? Might you want the same
locking there?
[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5174 bytes --]
next prev parent reply other threads:[~2021-12-08 14:15 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <87ft22dxop.fsf@nanos.tec.linutronix.de>
[not found] ` <27357c74bdc3b52bdf59e6f48cd8690495116d64.camel@infradead.org>
[not found] ` <877dnedt7l.fsf@nanos.tec.linutronix.de>
[not found] ` <87zh09tcqz.fsf@nanos.tec.linutronix.de>
2021-02-16 13:53 ` [PATCH] use x86 cpu park to speedup smp_init in kexec situation David Woodhouse
2021-02-16 15:10 ` David Woodhouse
2021-02-16 21:18 ` David Woodhouse
2021-12-08 14:14 ` David Woodhouse [this message]
2021-12-08 14:50 ` Paul E. McKenney
2021-12-08 15:10 ` David Woodhouse
2021-12-08 16:57 ` David Woodhouse
2021-12-08 17:35 ` Paul E. McKenney
2021-12-08 18:32 ` David Woodhouse
2021-12-08 19:03 ` Paul E. McKenney
2021-12-08 20:35 ` David Woodhouse
2021-12-08 21:09 ` Paul E. McKenney
2020-12-15 14:46 shenkai (D)
2020-12-15 16:31 ` Andy Lutomirski
2020-12-15 21:20 ` Thomas Gleixner
2020-12-16 8:45 ` shenkai (D)
2020-12-16 10:12 ` Thomas Gleixner
2020-12-16 14:18 ` shenkai (D)
2020-12-16 15:31 ` Thomas Gleixner
2020-12-17 14:53 ` shenkai (D)
2021-01-07 15:18 ` David Woodhouse
2021-01-19 12:12 ` David Woodhouse
2021-01-21 14:55 ` Thomas Gleixner
2021-01-21 15:42 ` David Woodhouse
2021-01-21 17:34 ` David Woodhouse
2021-02-01 10:36 ` David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d7939a95731de8b8eb9245c330f014772e40f145.camel@infradead.org \
--to=dwmw2@infradead.org \
--cc=bp@alien8.de \
--cc=hejingxian@huawei.com \
--cc=hewenliang4@huawei.com \
--cc=hpa@zytor.com \
--cc=hushiyuan@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luolongjun@huawei.com \
--cc=luto@kernel.org \
--cc=mimoja@amazon.com \
--cc=mingo@redhat.com \
--cc=paulmckrcu@gmail.com \
--cc=shenkai8@huawei.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).