From: Ingo Molnar <mingo@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>,
Sedat Dilek <sedat.dilek@gmail.com>,
Waiman Long <waiman.long@hp.com>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Jeff Layton <jlayton@redhat.com>,
Miklos Szeredi <mszeredi@suse.cz>, Ingo Molnar <mingo@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Steven Rostedt <rostedt@goodmis.org>,
Andi Kleen <andi@firstfloor.org>,
"Chandramouleeswaran, Aswin" <aswin@hp.com>,
"Norton, Scott J" <scott.norton@hp.com>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Arnaldo Carvalho de Melo <acme@infradead.org>
Subject: Re: [PATCH v7 1/4] spinlock: A new lockref structure for lockless update of refcount
Date: Mon, 2 Sep 2013 09:05:38 +0200 [thread overview]
Message-ID: <20130902070538.GA31639@gmail.com> (raw)
In-Reply-To: <CA+55aFyodR650Y8yXXenH1XHo-__5avng1WSK=AiSJx4mYy25g@mail.gmail.com>
* Linus Torvalds <torvalds@linux-foundation.org> wrote:
> On Sun, Sep 1, 2013 at 5:12 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > It *is* one of the few locked accesses remaining, and it's clearly
> > getting called a lot (three calls per system call: two mntput's - one
> > for the root path, one for the result path, and one from path_init ->
> > rcu_walk_init), but with up to 8% CPU time for basically that one
> > "lock xadd" instruction is damn odd. I can't see how that could happen
> > without seriously nasty cacheline bouncing, but I can't see how *that*
> > can happen when all the accesses seem to be from the current CPU.
>
> So, I wanted to double-check that "it can only be that expensive if
> there's cacheline bouncing" statement. Thinking "maybe it's just
> really expensive. Even when running just a single thread".
>
> So I set MAX_THREADS to 1 in my stupid benchmark, just to see what happens..
>
> And almost everything changes as expected: now we don't have any
> cacheline bouncing any more, so lockref_put_or_lock() and
> lockref_get_or_lock() no longer dominate - instead of being 20%+ each,
> they are now just 3%.
>
> What _didn't_ change? Right. lg_local_lock() is still 6.40%. Even when
> single-threaded. It's now the #1 function in my profile:
>
> 6.40% lg_local_lock
> 5.42% copy_user_enhanced_fast_string
> 5.14% sysret_check
> 4.79% link_path_walk
> 4.41% 0x00007ff861834ee3
> 4.33% avc_has_perm_flags
> 4.19% __lookup_mnt
> 3.83% lookup_fast
>
> (that "copy_user_enhanced_fast_string" is when we copy the "struct
> stat" from kernel space to user space)
>
> The instruction-level profile just looking like
>
> ??? ffffffff81078e70 <lg_local_lock>:
> 2.06 ??? push %rbp
> 1.06 ??? mov %rsp,%rbp
> 0.11 ??? mov (%rdi),%rdx
> 2.13 ??? add %gs:0xcd48,%rdx
> 0.92 ??? mov $0x100,%eax
> 85.87 ??? lock xadd %ax,(%rdx)
> 0.04 ??? movzbl %ah,%ecx
> ??? cmp %al,%cl
> 3.60 ??? ??? je 31
> ??? nop
> ???28: pause
> ??? movzbl (%rdx),%eax
> ??? cmp %cl,%al
> ??? ??? jne 28
> ???31: pop %rbp
> 4.22 ??? ??? retq
The Haswell perf code isn't very widely tested yet as it took quite some
time to get it ready for upstream and thus got merged late, but on its
face this looks like a pretty good profile.
With one detail:
> so that instruction sequence is just expensive, and it is expensive
> without any cacheline bouncing. The expense seems to be 100% simply due
> to the fact that it's an atomic serializing instruction, and it just
> gets called way too much.
>
> So lockref_[get|put]_or_lock() are each called once per pathname lookup
> (because the RCU accesses to the dentries get turned into a refcount,
> and then that refcount gets dropped). But lg_local_lock() gets called
> twice: once for path_init(), and once for mntput() - I think I was wrong
> about mntput getting called twice.
>
> So it doesn't seem to be cacheline bouncing at all. It's just
> "serializing instructions are really expensive" together with calling
> that function too much. And we've optimized pathname lookup so much that
> even a single locked instruction shows up like a sort thumb.
>
> I guess we should be proud.
It still looks anomalous to me, on fresh Intel hardware. One suggestion:
could you, just for pure testing purposes, turn HT off and do a quick
profile that way?
The XADD, even if it's all in the fast path, could be a pretty natural
point to 'yield' an SMT context on a given core, giving it artificially
high overhead.
Note that to test HT off an intrusive reboot is probably not needed, if
the HT siblings are right after each other in the CPU enumeration sequence
then you can turn HT "off" effectively by running the workload only on 4
cores:
taskset 0x55 ./my-test
and reducing the # of your workload threads to 4 or so.
Thanks,
Ingo
next prev parent reply other threads:[~2013-09-02 7:05 UTC|newest]
Thread overview: 151+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-08-06 3:12 [PATCH v7 0/4] Lockless update of reference count protected by spinlock Waiman Long
2013-08-06 3:12 ` [PATCH v7 1/4] spinlock: A new lockref structure for lockless update of refcount Waiman Long
2013-08-29 1:40 ` Linus Torvalds
2013-08-29 4:44 ` Benjamin Herrenschmidt
2013-08-29 7:00 ` Ingo Molnar
2013-08-29 16:43 ` Linus Torvalds
2013-08-29 19:25 ` Linus Torvalds
2013-08-29 23:42 ` Linus Torvalds
2013-08-30 0:26 ` Benjamin Herrenschmidt
2013-08-30 0:49 ` Linus Torvalds
2013-08-30 2:06 ` Michael Neuling
2013-08-30 2:30 ` Benjamin Herrenschmidt
2013-08-30 2:35 ` Linus Torvalds
2013-08-30 2:45 ` Benjamin Herrenschmidt
2013-08-30 2:31 ` Linus Torvalds
2013-08-30 2:43 ` Benjamin Herrenschmidt
2013-08-30 7:16 ` Ingo Molnar
2013-08-30 15:28 ` Linus Torvalds
2013-08-30 3:12 ` Waiman Long
2013-08-30 3:54 ` Linus Torvalds
2013-08-30 7:55 ` Sedat Dilek
2013-08-30 8:10 ` Sedat Dilek
2013-08-30 9:27 ` Sedat Dilek
2013-08-30 9:48 ` Ingo Molnar
2013-08-30 9:56 ` Sedat Dilek
2013-08-30 9:58 ` Sedat Dilek
2013-08-30 10:29 ` Sedat Dilek
2013-08-30 10:36 ` Peter Zijlstra
2013-08-30 10:44 ` Sedat Dilek
2013-08-30 10:46 ` Sedat Dilek
2013-08-30 10:52 ` Peter Zijlstra
2013-08-30 10:57 ` Sedat Dilek
2013-08-30 14:05 ` Sedat Dilek
2013-08-30 11:19 ` Sedat Dilek
2013-08-30 10:38 ` Sedat Dilek
2013-08-30 15:34 ` Linus Torvalds
2013-08-30 15:38 ` Sedat Dilek
2013-08-30 16:12 ` Steven Rostedt
2013-08-30 16:16 ` Sedat Dilek
2013-08-30 18:42 ` Linus Torvalds
2013-08-30 16:32 ` Linus Torvalds
2013-08-30 16:37 ` Sedat Dilek
2013-08-30 16:52 ` Linus Torvalds
2013-08-30 17:11 ` Sedat Dilek
2013-08-30 17:26 ` Linus Torvalds
2013-09-01 10:01 ` Sedat Dilek
2013-09-01 10:33 ` Sedat Dilek
2013-09-01 15:32 ` Linus Torvalds
2013-09-01 15:45 ` Sedat Dilek
2013-09-01 15:55 ` Linus Torvalds
2013-09-02 10:30 ` Sedat Dilek
2013-09-02 16:09 ` David Ahern
2013-09-01 20:59 ` Linus Torvalds
2013-09-01 21:23 ` Al Viro
2013-09-01 22:16 ` Linus Torvalds
2013-09-01 22:35 ` Al Viro
2013-09-01 22:44 ` Al Viro
2013-09-01 22:58 ` Linus Torvalds
2013-09-01 22:48 ` Linus Torvalds
2013-09-01 23:30 ` Al Viro
2013-09-02 0:12 ` Linus Torvalds
2013-09-02 0:50 ` Linus Torvalds
2013-09-02 7:05 ` Ingo Molnar [this message]
2013-09-02 16:44 ` Linus Torvalds
2013-09-03 10:15 ` Ingo Molnar
2013-09-03 15:41 ` Linus Torvalds
2013-09-03 18:34 ` Linus Torvalds
2013-09-03 19:19 ` Ingo Molnar
2013-09-03 21:05 ` Linus Torvalds
2013-09-03 21:13 ` Linus Torvalds
2013-09-03 21:34 ` Linus Torvalds
2013-09-03 21:39 ` Linus Torvalds
2013-09-03 14:08 ` Pavel Machek
2013-09-03 22:37 ` Sedat Dilek
2013-09-03 22:55 ` Dave Jones
2013-09-03 23:05 ` Sedat Dilek
2013-09-03 23:15 ` Dave Jones
2013-09-03 23:20 ` Sedat Dilek
2013-09-03 23:45 ` Sedat Dilek
2013-08-30 18:33 ` Waiman Long
2013-08-30 18:53 ` Linus Torvalds
2013-08-30 19:20 ` Waiman Long
2013-08-30 19:33 ` Linus Torvalds
2013-08-30 20:15 ` Waiman Long
2013-08-30 20:43 ` Linus Torvalds
2013-08-30 20:54 ` Al Viro
2013-08-30 21:03 ` Linus Torvalds
2013-08-30 21:44 ` Al Viro
2013-08-30 22:30 ` Linus Torvalds
2013-08-31 21:23 ` Al Viro
2013-08-31 22:49 ` Linus Torvalds
2013-08-31 23:27 ` Al Viro
2013-09-01 0:13 ` Al Viro
2013-09-01 17:48 ` Al Viro
2013-09-09 8:30 ` Peter Zijlstra
2013-08-30 21:10 ` Waiman Long
2013-08-30 21:22 ` Linus Torvalds
2013-08-30 21:30 ` Al Viro
2013-08-30 21:42 ` Waiman Long
2013-08-30 19:40 ` Al Viro
2013-08-30 19:52 ` Waiman Long
2013-08-30 20:26 ` Al Viro
2013-08-30 20:35 ` Waiman Long
2013-08-30 20:48 ` Al Viro
2013-08-31 2:02 ` Waiman Long
2013-08-31 2:35 ` Al Viro
2013-08-31 2:42 ` Al Viro
2013-09-02 19:25 ` Waiman Long
2013-09-03 6:01 ` Ingo Molnar
2013-09-03 7:24 ` Sedat Dilek
2013-09-03 15:38 ` Linus Torvalds
2013-09-03 15:14 ` Waiman Long
2013-09-03 15:34 ` Linus Torvalds
2013-09-03 19:09 ` Linus Torvalds
2013-09-03 21:01 ` Waiman Long
2013-09-04 14:52 ` Waiman Long
2013-09-04 15:14 ` Linus Torvalds
2013-09-04 19:25 ` Waiman Long
2013-09-04 21:34 ` Linus Torvalds
2013-09-05 2:35 ` Waiman Long
2013-09-05 13:31 ` Ingo Molnar
2013-09-05 17:33 ` Waiman Long
2013-09-05 17:40 ` Ingo Molnar
2013-09-03 22:41 ` Sedat Dilek
2013-09-03 23:11 ` Sedat Dilek
2013-09-08 21:45 ` Linus Torvalds
2013-09-09 0:03 ` Al Viro
2013-09-09 0:25 ` Linus Torvalds
2013-09-09 0:35 ` Al Viro
2013-09-09 0:38 ` Linus Torvalds
2013-09-09 0:57 ` Al Viro
2013-09-09 2:09 ` Ramkumar Ramachandra
2013-09-09 0:30 ` Al Viro
2013-09-09 3:32 ` Linus Torvalds
2013-09-09 4:06 ` Ramkumar Ramachandra
2013-09-09 5:44 ` Al Viro
2013-08-30 17:17 ` Peter Zijlstra
2013-08-30 17:28 ` Linus Torvalds
2013-08-30 17:33 ` Linus Torvalds
2013-08-29 15:20 ` Waiman Long
2013-08-06 3:12 ` [PATCH v7 2/4] spinlock: Enable x86 architecture to do lockless refcount update Waiman Long
2013-08-06 3:12 ` [PATCH v7 3/4] dcache: replace d_lock/d_count by d_lockcnt Waiman Long
2013-08-06 3:12 ` [PATCH v7 4/4] dcache: Enable lockless update of dentry's refcount Waiman Long
2013-08-13 18:03 ` [PATCH v7 0/4] Lockless update of reference count protected by spinlock Waiman Long
2013-08-31 3:06 [PATCH v7 1/4] spinlock: A new lockref structure for lockless update of refcount George Spelvin
2013-08-31 17:16 ` Linus Torvalds
2013-09-01 8:50 ` George Spelvin
2013-09-01 11:10 ` Theodore Ts'o
2013-09-01 15:49 ` Linus Torvalds
2013-09-01 18:11 ` Steven Rostedt
2013-09-01 20:03 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130902070538.GA31639@gmail.com \
--to=mingo@kernel.org \
--cc=a.p.zijlstra@chello.nl \
--cc=acme@infradead.org \
--cc=andi@firstfloor.org \
--cc=aswin@hp.com \
--cc=benh@kernel.crashing.org \
--cc=jlayton@redhat.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=mszeredi@suse.cz \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=scott.norton@hp.com \
--cc=sedat.dilek@gmail.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
--cc=waiman.long@hp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).