From: Ingo Molnar <mingo@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
David Laight <David.Laight@ACULAB.COM>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"netfilter-devel@vger.kernel.org"
<netfilter-devel@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"oleg@redhat.com" <oleg@redhat.com>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"mingo@redhat.com" <mingo@redhat.com>,
"dave@stgolabs.net" <dave@stgolabs.net>,
"manfred@colorfullife.com" <manfred@colorfullife.com>,
"tj@kernel.org" <tj@kernel.org>, "arnd@arndb.de" <arnd@arndb.de>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
"will.deacon@arm.com" <will.deacon@arm.com>,
"stern@rowland.harvard.edu" <stern@rowland.harvard.edu>,
"parri.andrea@gmail.com" <parri.andrea@gmail.com>,
"torvalds@linux-foundation.org" <torvalds@linux-foundation.org>
Subject: Re: [PATCH v2 0/9] Remove spin_unlock_wait()
Date: Fri, 7 Jul 2017 10:31:28 +0200 [thread overview]
Message-ID: <20170707083128.wqk6msuuhtyykhpu@gmail.com> (raw)
In-Reply-To: <20170706165036.v4u5rbz56si4emw5@hirez.programming.kicks-ass.net>
* Peter Zijlstra <peterz@infradead.org> wrote:
> On Thu, Jul 06, 2017 at 09:20:24AM -0700, Paul E. McKenney wrote:
> > On Thu, Jul 06, 2017 at 06:05:55PM +0200, Peter Zijlstra wrote:
> > > On Thu, Jul 06, 2017 at 02:12:24PM +0000, David Laight wrote:
> > > > From: Paul E. McKenney
> >
> > [ . . . ]
> >
> > > Now on the one hand I feel like Oleg that it would be a shame to loose
> > > the optimization, OTOH this thing is really really tricky to use,
> > > and has lead to a number of bugs already.
> >
> > I do agree, it is a bit sad to see these optimizations go. So, should
> > this make mainline, I will be tagging the commits that spin_unlock_wait()
> > so that they can be easily reverted should someone come up with good
> > semantics and a compelling use case with compelling performance benefits.
>
> Ha!, but what would constitute 'good semantics' ?
>
> The current thing is something along the lines of:
>
> "Waits for the currently observed critical section
> to complete with ACQUIRE ordering such that it will observe
> whatever state was left by said critical section."
>
> With the 'obvious' benefit of limited interference on those actually
> wanting to acquire the lock, and a shorter wait time on our side too,
> since we only need to wait for completion of the current section, and
> not for however many contender are before us.
There's another, probably just as significant advantage: queued_spin_unlock_wait()
is 'read-only', while spin_lock()+spin_unlock() dirties the lock cache line. On
any bigger system this should make a very measurable difference - if
spin_unlock_wait() is ever used in a performance critical code path.
> Not sure I have an actual (micro) benchmark that shows a difference
> though.
It should be pretty obvious from pretty much any profile, the actual lock+unlock
sequence that modifies the lock cache line is essentially a global cacheline
bounce.
> Is this all good enough to retain the thing, I dunno. Like I said, I'm
> conflicted on the whole thing. On the one hand its a nice optimization, on the
> other hand I don't want to have to keep fixing these bugs.
So on one hand it's _obvious_ that spin_unlock_wait() is both faster on the local
_and_ the remote CPUs for any sort of use case where performance matters - I don't
even understand how that can be argued otherwise.
The real question, does any use-case (we care about) exist.
Here's a quick list of all the use cases:
net/netfilter/nf_conntrack_core.c:
- This is I believe the 'original', historic spin_unlock_wait() usecase that
still exists in the kernel. spin_unlock_wait() is only used in a rare case,
when the netfilter hash is resized via nf_conntrack_hash_resize() - which is
a very heavy operation to begin with. It will no doubt get slower with the
proposed changes, but it probably does not matter. A networking person
Acked-by would be nice though.
drivers/ata/libata-eh.c:
- Locking of the ATA port in ata_scsi_cmd_error_handler(), presumably this can
race with IRQs and ioctls() on other CPUs. Very likely not performance
sensitive in any fashion, on IO errors things stop for many seconds anyway.
ipc/sem.c:
- A rare race condition branch in the SysV IPC semaphore freeing code in
exit_sem() - where even the main code flow is not performance sensitive,
because typical database workloads get their semaphore arrays during startup
and don't ever do heavy runtime allocation/freeing of them.
kernel/sched/completion.c:
- completion_done(). This is actually a (comparatively) rarely used completion
API call - almost all the upstream usecases are in drivers, plus two in
filesystems - neither usecase seems in a performance critical hot path.
Completions typically involve scheduling and context switching, so in the
worst case the proposed change adds overhead to a scheduling slow path.
So I'd argue that unless there's some surprising performance aspect of a
completion_done() user, the proposed changes should not cause any performance
trouble.
In fact I'd argue that any future high performance spin_unlock_wait() user is
probably better off open coding the unlock-wait poll loop (and possibly thinking
hard about eliminating it altogether). If such patterns pop up in the kernel we
can think about consolidating them into a single read-only primitive again.
I.e. I think the proposed changes are doing no harm, and the unavailability of a
generic primitive does not hinder future optimizations either in any significant
fashion.
Thanks,
Ingo
next prev parent reply other threads:[~2017-07-07 8:31 UTC|newest]
Thread overview: 122+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-29 23:59 [PATCH RFC 0/26] Remove spin_unlock_wait() Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 01/26] netfilter: Replace spin_unlock_wait() with lock/unlock pair Paul E. McKenney
[not found] ` <a6642feb-2f3a-980f-5ed6-2deb79563e6b@colorfullife.com>
2017-07-02 2:00 ` Paul E. McKenney
2017-07-03 14:39 ` Alan Stern
2017-07-03 17:14 ` Paul E. McKenney
2017-07-03 19:01 ` Manfred Spraul
2017-07-03 19:57 ` Alan Stern
2017-07-06 18:43 ` Manfred Spraul
2017-07-03 20:04 ` Alan Stern
2017-07-03 20:53 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 02/26] task_work: " Paul E. McKenney
2017-06-30 11:04 ` Oleg Nesterov
2017-06-30 12:50 ` Paul E. McKenney
2017-06-30 15:20 ` Oleg Nesterov
2017-06-30 16:16 ` Paul E. McKenney
2017-06-30 17:21 ` Paul E. McKenney
2017-06-30 19:21 ` Oleg Nesterov
2017-06-30 19:50 ` Alan Stern
2017-06-30 20:04 ` Paul E. McKenney
2017-06-30 20:02 ` Paul E. McKenney
2017-06-30 20:19 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 03/26] sched: " Paul E. McKenney
2017-06-30 10:31 ` Arnd Bergmann
2017-06-30 12:35 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 04/26] completion: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 05/26] exit: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 07/26] drivers/ata: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 08/26] locking: Remove spin_unlock_wait() generic definitions Paul E. McKenney
2017-06-30 9:19 ` Will Deacon
2017-06-30 12:38 ` Paul E. McKenney
2017-06-30 13:13 ` Will Deacon
2017-06-30 22:18 ` Paul E. McKenney
2017-07-03 13:15 ` Will Deacon
2017-07-03 16:18 ` Paul E. McKenney
2017-07-03 16:40 ` Linus Torvalds
2017-07-03 17:13 ` Will Deacon
2017-07-03 22:30 ` Paul E. McKenney
2017-07-03 22:49 ` Linus Torvalds
2017-07-04 0:39 ` Paul E. McKenney
2017-07-04 0:54 ` Paul E. McKenney
2017-07-03 21:10 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 09/26] alpha: Remove spin_unlock_wait() arch-specific definitions Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 10/26] arc: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 11/26] arm: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 12/26] arm64: " Paul E. McKenney
2017-06-30 9:20 ` Will Deacon
2017-06-30 17:29 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 13/26] blackfin: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 14/26] hexagon: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 15/26] ia64: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 16/26] m32r: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 18/26] mips: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 19/26] mn10300: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 20/26] parisc: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 21/26] powerpc: " Paul E. McKenney
2017-07-02 3:58 ` Boqun Feng
2017-07-05 23:57 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 22/26] s390: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 23/26] sh: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 24/26] sparc: " Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 25/26] tile: " Paul E. McKenney
2017-06-30 0:06 ` Linus Torvalds
2017-06-30 0:09 ` Paul E. McKenney
2017-06-30 0:14 ` Paul E. McKenney
2017-06-30 0:10 ` Linus Torvalds
2017-06-30 0:24 ` Paul E. McKenney
2017-06-30 0:01 ` [PATCH RFC 26/26] xtensa: " Paul E. McKenney
[not found] ` <1498780894-8253-6-git-send-email-paulmck@linux.vnet.ibm.com>
2017-07-01 19:23 ` [PATCH RFC 06/26] ipc: Replace spin_unlock_wait() with lock/unlock pair Manfred Spraul
2017-07-02 3:16 ` Paul E. McKenney
2017-07-05 23:29 ` [PATCH v2 0/9] Remove spin_unlock_wait() Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 1/9] net/netfilter/nf_conntrack_core: Fix net_conntrack_lock() Paul E. McKenney
2017-07-06 18:45 ` Manfred Spraul
2017-07-06 20:26 ` Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 2/9] task_work: Replace spin_unlock_wait() with lock/unlock pair Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 3/9] sched: " Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 4/9] completion: " Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 5/9] exit: " Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 6/9] ipc: " Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 7/9] drivers/ata: " Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 8/9] locking: Remove spin_unlock_wait() generic definitions Paul E. McKenney
2017-07-05 23:31 ` [PATCH v2 9/9] arch: Remove spin_unlock_wait() arch-specific definitions Paul E. McKenney
2017-07-06 14:12 ` [PATCH v2 0/9] Remove spin_unlock_wait() David Laight
2017-07-06 15:21 ` Paul E. McKenney
2017-07-06 16:10 ` Peter Zijlstra
2017-07-06 16:24 ` Paul E. McKenney
2017-07-06 16:41 ` Peter Zijlstra
2017-07-06 17:03 ` Paul E. McKenney
2017-07-06 16:49 ` Alan Stern
2017-07-06 16:54 ` Peter Zijlstra
2017-07-06 19:37 ` Alan Stern
2017-07-06 16:05 ` Peter Zijlstra
2017-07-06 16:20 ` Paul E. McKenney
2017-07-06 16:50 ` Peter Zijlstra
2017-07-06 17:08 ` Will Deacon
2017-07-06 17:29 ` Paul E. McKenney
2017-07-06 17:18 ` Paul E. McKenney
2017-07-07 8:31 ` Ingo Molnar [this message]
2017-07-07 8:44 ` Peter Zijlstra
2017-07-07 10:33 ` Ingo Molnar
2017-07-07 11:23 ` Peter Zijlstra
2017-07-07 14:41 ` Paul E. McKenney
2017-07-08 8:43 ` Ingo Molnar
2017-07-08 11:41 ` Paul E. McKenney
2017-07-07 17:47 ` Manfred Spraul
2017-07-08 8:35 ` Ingo Molnar
2017-07-08 11:39 ` Paul E. McKenney
2017-07-08 12:30 ` Ingo Molnar
2017-07-08 14:45 ` Paul E. McKenney
2017-07-08 16:21 ` Alan Stern
2017-07-10 17:22 ` Manfred Spraul
2017-07-07 8:06 ` Ingo Molnar
2017-07-07 9:32 ` Ingo Molnar
2017-07-07 19:27 ` [PATCH v3 " Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 1/9] net/netfilter/nf_conntrack_core: Fix net_conntrack_lock() Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 2/9] task_work: Replace spin_unlock_wait() with lock/unlock pair Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 3/9] sched: " Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 4/9] completion: " Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 5/9] exit: " Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 6/9] ipc: " Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 7/9] drivers/ata: " Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 8/9] locking: Remove spin_unlock_wait() generic definitions Paul E. McKenney
2017-07-07 19:28 ` [PATCH v3 9/9] arch: Remove spin_unlock_wait() arch-specific definitions Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170707083128.wqk6msuuhtyykhpu@gmail.com \
--to=mingo@kernel.org \
--cc=David.Laight@ACULAB.COM \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=dave@stgolabs.net \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=manfred@colorfullife.com \
--cc=mingo@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=netfilter-devel@vger.kernel.org \
--cc=oleg@redhat.com \
--cc=parri.andrea@gmail.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=stern@rowland.harvard.edu \
--cc=tj@kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).