From: Dave Hansen <dave.hansen@linux.intel.com>
To: Linus Torvalds <torvalds@linux-foundation.org>,
"Theodore Ts'o" <tytso@mit.edu>,
Andrew Morton <akpm@linux-foundation.org>,
"linux-ext4@vger.kernel.org" <linux-ext4@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"H. Peter Anvin" <hpa@linux.intel.com>
Subject: Re: [REGRESSION] 998ef75ddb and aio-dio-invalidate-failure w/ data=journal
Date: Mon, 5 Oct 2015 09:23:15 -0700 [thread overview]
Message-ID: <5612A3F3.2040609@linux.intel.com> (raw)
In-Reply-To: <CA+55aFzARo_ZtbO6PDxgenWQtEEbynBCWFWCwVJT2NbXmJOd9Q@mail.gmail.com>
On 10/05/2015 08:58 AM, Linus Torvalds wrote:
...
> Dave, mind sharing the micro-benchmark or perhaps even just a kernel
> profile of it? How is that "iov_iter_fault_in_readable()" so
> noticeable? It really shouldn't be a big deal.
The micro was just plugging this test:
https://www.sr71.net/~dave/intel/write1byte.c
In to will-it-scale:
https://github.com/antonblanchard/will-it-scale
iov_iter_fault_in_readable() shows up as the third-most expensive kernel
function in a profile:
> 7.45% write1byte_proc [kernel.kallsyms] [k] copy_user_enhanced_fast_string
> 6.51% write1byte_proc [kernel.kallsyms] [k] unlock_page
> 6.04% write1byte_proc [kernel.kallsyms] [k] iov_iter_fault_in_readable
> 5.23% write1byte_proc libc-2.20.so [.] __GI___libc_write
> 4.86% write1byte_proc [kernel.kallsyms] [k] entry_SYSCALL_64
> 4.48% write1byte_proc [kernel.kallsyms] [k] iov_iter_copy_from_user_atomic
> 3.94% write1byte_proc [kernel.kallsyms] [k] generic_perform_write
> 3.74% write1byte_proc [kernel.kallsyms] [k] mutex_lock
> 3.59% write1byte_proc [kernel.kallsyms] [k] entry_SYSCALL_64_after_swapgs
> 3.55% write1byte_proc [kernel.kallsyms] [k] find_get_entry
> 3.53% write1byte_proc [kernel.kallsyms] [k] vfs_write
> 3.17% write1byte_proc [kernel.kallsyms] [k] find_lock_entry
> 3.17% write1byte_proc [kernel.kallsyms] [k] put_page
The disassembly points at the stac/clac pair being the culprits inside
the function (copy/paste from 'perf top' disassebly here):
...
> │ stac
> 24.57 │ mov (%rcx),%sil
> 15.70 │ clac
> 28.77 │ test %eax,%eax
> 2.15 │ mov %sil,-0x1(%rbp)
> 8.93 │ ↓ jne 66
> 2.31 │ movslq %edx,%rdx
One thing I've been noticing on Skylake is that barriers (implicit and
explicit) are showing up more in profiles. What we're seeing here
probably isn't actually stac/clac overhead, but the cost of finishing
some other operations that are outstanding before we can proceed through
here.
next prev parent reply other threads:[~2015-10-05 16:23 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-05 15:22 [REGRESSION] 998ef75ddb and aio-dio-invalidate-failure w/ data=journal Theodore Ts'o
2015-10-05 15:58 ` Linus Torvalds
2015-10-05 16:23 ` Dave Hansen [this message]
2015-10-05 20:22 ` Linus Torvalds
2015-10-05 20:48 ` Dave Hansen
2015-10-05 21:18 ` Linus Torvalds
2015-10-05 21:55 ` Linus Torvalds
2015-10-05 23:33 ` Dave Hansen
2015-10-06 9:01 ` Linus Torvalds
2015-10-05 20:49 ` H. Peter Anvin
2015-10-06 7:56 ` Ingo Molnar
2015-10-06 9:10 ` Linus Torvalds
2015-10-06 9:27 ` Ingo Molnar
2015-10-06 13:29 ` Linus Torvalds
2015-10-06 13:42 ` Ingo Molnar
2015-10-05 16:03 ` Dave Hansen
2015-10-05 18:04 ` Dave Hansen
2015-10-07 3:34 ` Theodore Ts'o
2015-10-07 7:32 ` Linus Torvalds
2015-10-07 15:43 ` Theodore Ts'o
2015-10-09 4:01 ` [PATCH] ext4: use private version of page_zero_new_buffers() for data=journal mode Theodore Ts'o
2015-10-13 6:06 ` Leonid V. Fedorenchik
2015-10-15 11:17 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5612A3F3.2040609@linux.intel.com \
--to=dave.hansen@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=hpa@linux.intel.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).