From: Peter Zijlstra <peterz@infradead.org>
To: Alex Kogan <alex.kogan@oracle.com>
Cc: Waiman Long <longman@redhat.com>,
linux@armlinux.org.uk, mingo@redhat.com, will.deacon@arm.com,
arnd@arndb.de, linux-arch@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de,
hpa@zytor.com, x86@kernel.org, steven.sistare@oracle.com,
daniel.m.jordan@oracle.com, dave.dice@oracle.com,
rahul.x.yadav@oracle.com
Subject: Re: [PATCH v2 3/5] locking/qspinlock: Introduce CNA into the slow path of qspinlock
Date: Wed, 3 Apr 2019 18:01:12 +0200 [thread overview]
Message-ID: <20190403160112.GK4038@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <6AEDE4F2-306A-4DF9-9307-9E3517C68A2B@oracle.com>
On Wed, Apr 03, 2019 at 11:39:09AM -0400, Alex Kogan wrote:
> >> The patch that I am looking for is to have a separate
> >> numa_queued_spinlock_slowpath() that coexists with
> >> native_queued_spinlock_slowpath() and
> >> paravirt_queued_spinlock_slowpath(). At boot time, we select the most
> >> appropriate one for the system at hand.
> Is this how this selection works today for paravirt?
> I see a PARAVIRT_SPINLOCKS config option, but IIUC you are talking about a different mechanism here.
> Can you, please, elaborate or give me a link to a page that explains that?
Oh man, you ask us to explain how paravirt patching works... that's
magic :-)
Basically, the compiler will emit a bunch of indirect calls to the
various pv_ops.*.* functions.
Then, at alternative_instructions() <- apply_paravirt() it will rewrite
all these indirect calls to direct calls to the function pointers that
are in the pv_ops structure at that time (+- more magic).
So we initialize the pv_ops.lock.* methods to the normal
native_queued_spin*() stuff, if KVM/Xen/whatever setup detectors pv
spnlock support changes the methods to the paravirt_queued_*() stuff.
If you wnt more details, you'll just have to read
arch/x86/include/asm/paravirt*.h and arch/x86/kernel/paravirt*.c, I
don't think there's a coherent writeup of all that.
> > Agreed; and until we have static_call, I think we can abuse the paravirt
> > stuff for this.
> >
> > By the time we patch the paravirt stuff:
> >
> > check_bugs()
> > alternative_instructions()
> > apply_paravirt()
> >
> > we should already have enumerated the NODE topology and so nr_node_ids()
> > should be set.
> >
> > So if we frob pv_ops.lock.queued_spin_lock_slowpath to
> > numa_queued_spin_lock_slowpath before that, it should all get patched
> > just right.
> >
> > That of course means the whole NUMA_AWARE_SPINLOCKS thing depends on
> > PARAVIRT_SPINLOCK, which is a bit awkward…
> Just to mention here, the patch so far does not address paravirt, but
> our goal is to add this support once we address all the concerns for
> the native version. So we will end up with four variants for the
> queued_spinlock_slowpath() — one for each combination of
> native/paravirt and NUMA/non-NUMA. Or perhaps we do not need a
> NUMA/paravirt variant?
I wouldn't bother with a pv version of the numa aware code at all. If
you have overcommitted guests, topology is likely irrelevant anyway. If
you have 1:1 pinned guests, they'll not use pv spinlocks anyway.
So keep it to tertiary choice:
- native
- native/numa
- paravirt
next prev parent reply other threads:[~2019-04-03 16:01 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-29 15:20 [PATCH v2 0/5] Add NUMA-awareness to qspinlock Alex Kogan
2019-03-29 15:20 ` [PATCH v2 1/5] locking/qspinlock: Make arch_mcs_spin_unlock_contended more generic Alex Kogan
2019-03-29 15:20 ` [PATCH v2 2/5] locking/qspinlock: Refactor the qspinlock slow path Alex Kogan
2019-03-29 15:20 ` [PATCH v2 3/5] locking/qspinlock: Introduce CNA into the slow path of qspinlock Alex Kogan
2019-04-01 9:06 ` Peter Zijlstra
2019-04-01 9:33 ` Peter Zijlstra
2019-04-03 15:53 ` Alex Kogan
2019-04-03 16:10 ` Peter Zijlstra
2019-04-01 9:21 ` Peter Zijlstra
2019-04-01 14:36 ` Waiman Long
2019-04-02 9:43 ` Peter Zijlstra
2019-04-03 15:39 ` Alex Kogan
2019-04-03 15:48 ` Waiman Long
2019-04-03 16:01 ` Peter Zijlstra [this message]
2019-04-04 5:05 ` Juergen Gross
2019-04-04 9:38 ` Peter Zijlstra
2019-04-04 18:03 ` Waiman Long
2019-06-04 23:21 ` Alex Kogan
2019-06-05 20:40 ` Peter Zijlstra
2019-06-06 15:21 ` Alex Kogan
2019-06-06 15:32 ` Waiman Long
2019-06-06 15:42 ` Waiman Long
2019-04-03 16:33 ` Waiman Long
2019-04-03 17:16 ` Peter Zijlstra
2019-04-03 17:40 ` Waiman Long
2019-04-04 2:02 ` Hanjun Guo
2019-04-04 3:14 ` Alex Kogan
2019-06-11 4:22 ` liwei (GF)
2019-06-12 4:38 ` Alex Kogan
2019-06-12 15:05 ` Waiman Long
2019-03-29 15:20 ` [PATCH v2 4/5] locking/qspinlock: Introduce starvation avoidance into CNA Alex Kogan
2019-04-02 10:37 ` Peter Zijlstra
2019-04-03 17:06 ` Alex Kogan
2019-03-29 15:20 ` [PATCH v2 5/5] locking/qspinlock: Introduce the shuffle reduction optimization " Alex Kogan
2019-04-01 9:09 ` [PATCH v2 0/5] Add NUMA-awareness to qspinlock Peter Zijlstra
2019-04-03 17:13 ` Alex Kogan
2019-07-03 11:58 ` Jan Glauber
2019-07-12 8:12 ` Hanjun Guo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190403160112.GK4038@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=alex.kogan@oracle.com \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=daniel.m.jordan@oracle.com \
--cc=dave.dice@oracle.com \
--cc=hpa@zytor.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=longman@redhat.com \
--cc=mingo@redhat.com \
--cc=rahul.x.yadav@oracle.com \
--cc=steven.sistare@oracle.com \
--cc=tglx@linutronix.de \
--cc=will.deacon@arm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).