From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Minchan Kim <minchan.kim@gmail.com>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Peter Zijlstra <peterz@infradead.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
cl@linux-foundation.org,
"hugh.dickins" <hugh.dickins@tiscali.co.uk>,
Nick Piggin <nickpiggin@yahoo.com.au>,
Ingo Molnar <mingo@elte.hu>
Subject: Re: [RFC][PATCH 6/8] mm: handle_speculative_fault()
Date: Wed, 6 Jan 2010 11:52:33 +0900 [thread overview]
Message-ID: <20100106115233.5621bd5e.kamezawa.hiroyu@jp.fujitsu.com> (raw)
In-Reply-To: <alpine.LFD.2.00.1001051718100.3630@localhost.localdomain>
On Tue, 5 Jan 2010 17:37:08 -0800 (PST)
Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
>
> On Wed, 6 Jan 2010, KAMEZAWA Hiroyuki wrote:
> >
> > I think this is the 1st reason but haven't rewrote rwsem itself and tested,
> > sorry.
>
> Here's a totally untested patch! It may or may not work. It builds for me,
> but that may be some cosmic accident. I _think_ I got the callee-clobbered
> register set right, but somebody should check the comment in the new
> rwsem_64.S, and double-check that the code actually matches what I tried
> to do.
>
> I had to change the inline asm to get the register sizes right too, so for
> all I know this screws up x86-32 too.
>
> In other words: UNTESTED! It may molest your pets and drink all your beer.
> You have been warned.
>
Thank you for warning ;)
My host boots successfully. Here is the result.
Result of Linus's rwmutex XADD patch.
Test:
while (1) {
touch memory
barrier
fork()->exit() if cpu==0
berrier
}
# Samples: 1121655736712
#
# Overhead Command Shared Object Symbol
# ........ ............... ........................ ......
#
50.26% multi-fault-all [kernel] [k] smp_invalidate_interrup
15.94% multi-fault-all [kernel] [k] flush_tlb_others_ipi
6.50% multi-fault-all [kernel] [k] intel_pmu_enable_all
3.17% multi-fault-all [kernel] [k] down_read_trylock
2.08% multi-fault-all [kernel] [k] do_wp_page
1.69% multi-fault-all [kernel] [k] page_fault
1.63% multi-fault-all ./multi-fault-all-fork [.] worker
1.53% multi-fault-all [kernel] [k] up_read
1.35% multi-fault-all [kernel] [k] do_page_fault
1.24% multi-fault-all [kernel] [k] _raw_spin_lock
1.10% multi-fault-all [kernel] [k] flush_tlb_page
0.96% multi-fault-all [kernel] [k] invalidate_interrupt0
0.92% multi-fault-all [kernel] [k] invalidate_interrupt3
0.90% multi-fault-all [kernel] [k] invalidate_interrupt2
Test:
while (1) {
touch memory
barrier
madvice DONTNEED to locally touched memory.
barrier
}
# Samples: 1335012531823
#
# Overhead Command Shared Object Symbol
# ........ ............... ........................ ......
#
32.17% multi-fault-all [kernel] [k] clear_page_c
9.60% multi-fault-all [kernel] [k] _raw_spin_lock
8.14% multi-fault-all [kernel] [k] _raw_spin_lock_irqsave
6.23% multi-fault-all [kernel] [k] down_read_trylock
4.98% multi-fault-all [kernel] [k] _raw_spin_lock_irq
4.63% multi-fault-all [kernel] [k] __mem_cgroup_try_charge
4.45% multi-fault-all [kernel] [k] up_read
3.83% multi-fault-all [kernel] [k] handle_mm_fault
3.19% multi-fault-all [kernel] [k] __rmqueue
3.05% multi-fault-all [kernel] [k] __mem_cgroup_commit_cha
2.39% multi-fault-all [kernel] [k] bad_range
1.78% multi-fault-all [kernel] [k] page_fault
1.74% multi-fault-all [kernel] [k] mem_cgroup_charge_commo
1.71% multi-fault-all [kernel] [k] lookup_page_cgroup
Then, the result is much improved by XADD rwsem.
In above profile, rwsem is still there.
But page-fault/sec is good. I hope some "big" machine users join to the test.
(I hope 4 sockets, at least..)
Here is peformance counter result of DONTNEED test. Counting the number of page
faults in 60 sec. So, bigger number of page fault is better.
[XADD rwsem]
[root@bluextal memory]# /root/bin/perf stat -e page-faults,cache-misses --repeat 5 ./multi-fault-all 8
Performance counter stats for './multi-fault-all 8' (5 runs):
41950863 page-faults ( +- 1.355% )
502983592 cache-misses ( +- 0.628% )
60.002682206 seconds time elapsed ( +- 0.000% )
[my patch]
[root@bluextal memory]# /root/bin/perf stat -e page-faults,cache-misses --repeat 5 ./multi-fault-all 8
Performance counter stats for './multi-fault-all 8' (5 runs):
35835485 page-faults ( +- 0.257% )
511445661 cache-misses ( +- 0.770% )
60.004243198 seconds time elapsed ( +- 0.002% )
Ah....xadd-rwsem seems to be faster than my patch ;)
Maybe my patch adds some big overhead (see below)
Then, on my host, I can get enough page-fault throughput by modifing rwsem.
Just for my interest, profile on my patch is here.
24.69% multi-fault-all [kernel] [k] clear_page_c
20.26% multi-fault-all [kernel] [k] _raw_spin_lock
8.59% multi-fault-all [kernel] [k] _raw_spin_lock_irq
4.88% multi-fault-all [kernel] [k] page_add_new_anon_rmap
4.33% multi-fault-all [kernel] [k] _raw_spin_lock_irqsave
4.27% multi-fault-all [kernel] [k] vma_put
3.55% multi-fault-all [kernel] [k] __mem_cgroup_try_charge
3.36% multi-fault-all [kernel] [k] find_vma_speculative
2.90% multi-fault-all [kernel] [k] handle_mm_fault
2.77% multi-fault-all [kernel] [k] __rmqueue
2.49% multi-fault-all [kernel] [k] bad_range
Hmm...spinlock contention is twice bigger.....????
20.46% multi-fault-all [kernel] [k] _raw_spin_lock
|
--- _raw_spin_lock
|
|--81.42%-- free_pcppages_bulk
| free_hot_cold_page
| __pagevec_free
| release_pages
| free_pages_and_swap_cache
| |
| |--99.57%-- unmap_vmas
| | zap_page_range
| | sys_madvise
| | system_call_fastpath
| | 0x3f6b0e2cf7
| --0.43%-- [...]
|
|--17.86%-- get_page_from_freelist
| __alloc_pages_nodemask
| handle_mm_fault
| do_page_fault
| page_fault
| 0x400940
| (nil)
--0.71%-- [...]
This seems to be page allocator lock. Hmm...why this big..
Thanks,
-Kame
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2010-01-06 2:55 UTC|newest]
Thread overview: 121+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-04 18:24 [RFC][PATCH 0/8] Speculative pagefault -v3 Peter Zijlstra
2010-01-04 18:24 ` [RFC][PATCH 1/8] mm: Remove pte reference from fault path Peter Zijlstra
2010-01-04 18:24 ` [RFC][PATCH 2/8] mm: Speculative pagefault infrastructure Peter Zijlstra
2010-01-04 18:24 ` [RFC][PATCH 3/8] mm: Add vma sequence count Peter Zijlstra
2010-01-04 18:24 ` [RFC][PATCH 4/8] mm: RCU free vmas Peter Zijlstra
2010-01-05 2:43 ` Paul E. McKenney
2010-01-05 8:28 ` Peter Zijlstra
2010-01-05 16:05 ` Paul E. McKenney
2010-01-04 18:24 ` [RFC][PATCH 5/8] mm: Speculative pte_map_lock() Peter Zijlstra
2010-01-04 18:24 ` [RFC][PATCH 6/8] mm: handle_speculative_fault() Peter Zijlstra
2010-01-05 0:25 ` KAMEZAWA Hiroyuki
2010-01-05 3:13 ` Linus Torvalds
2010-01-05 8:17 ` Peter Zijlstra
2010-01-05 8:57 ` Peter Zijlstra
2010-01-05 15:34 ` Linus Torvalds
2010-01-05 15:40 ` Al Viro
2010-01-05 16:10 ` Linus Torvalds
2010-01-06 15:41 ` Peter Zijlstra
2010-01-05 9:37 ` Peter Zijlstra
2010-01-05 23:35 ` Linus Torvalds
2010-01-05 4:29 ` Minchan Kim
2010-01-05 4:43 ` KAMEZAWA Hiroyuki
2010-01-05 5:10 ` Linus Torvalds
2010-01-05 5:30 ` KAMEZAWA Hiroyuki
2010-01-05 7:39 ` KAMEZAWA Hiroyuki
2010-01-05 15:26 ` Linus Torvalds
2010-01-05 16:14 ` Linus Torvalds
2010-01-05 17:25 ` Andi Kleen
2010-01-05 17:47 ` Christoph Lameter
2010-01-05 18:00 ` Andi Kleen
2010-01-05 17:55 ` Linus Torvalds
2010-01-05 18:13 ` Christoph Lameter
2010-01-05 18:25 ` Linus Torvalds
2010-01-05 18:46 ` Christoph Lameter
2010-01-05 18:56 ` Linus Torvalds
2010-01-05 19:15 ` Christoph Lameter
2010-01-05 19:28 ` Linus Torvalds
2010-01-05 18:55 ` Paul E. McKenney
2010-01-05 19:08 ` Linus Torvalds
2010-01-05 19:23 ` Paul E. McKenney
2010-01-05 20:29 ` Peter Zijlstra
2010-01-05 20:46 ` Linus Torvalds
2010-01-05 21:00 ` Linus Torvalds
2010-01-05 23:29 ` Paul E. McKenney
2010-01-06 0:22 ` KAMEZAWA Hiroyuki
2010-01-06 1:37 ` Linus Torvalds
2010-01-06 2:52 ` KAMEZAWA Hiroyuki [this message]
2010-01-06 3:27 ` Linus Torvalds
2010-01-06 3:56 ` KAMEZAWA Hiroyuki
2010-01-06 4:20 ` Linus Torvalds
2010-01-06 7:06 ` KAMEZAWA Hiroyuki
2010-01-06 7:49 ` Minchan Kim
2010-01-06 9:39 ` Linus Torvalds
2010-01-07 1:00 ` KAMEZAWA Hiroyuki
2010-01-08 16:53 ` Peter Zijlstra
2010-01-08 17:22 ` Linus Torvalds
2010-01-08 17:43 ` Christoph Lameter
2010-01-08 17:52 ` Linus Torvalds
2010-01-08 18:33 ` Christoph Lameter
2010-01-08 18:46 ` Andi Kleen
2010-01-08 18:56 ` Christoph Lameter
2010-01-08 19:10 ` Andi Kleen
2010-01-08 19:11 ` Linus Torvalds
2010-01-08 19:28 ` Andi Kleen
2010-01-08 19:39 ` Linus Torvalds
2010-01-08 19:42 ` Linus Torvalds
2010-01-08 21:36 ` Linus Torvalds
2010-01-08 21:46 ` Christoph Lameter
2010-01-08 22:43 ` Linus Torvalds
2010-01-08 22:43 ` Linus Torvalds
2010-01-09 14:47 ` Ed Tomlinson
2010-01-10 5:27 ` Nitin Gupta
2010-01-05 15:14 ` Christoph Lameter
2010-01-05 8:18 ` Peter Zijlstra
2010-01-05 6:00 ` Minchan Kim
2010-01-05 4:48 ` Linus Torvalds
2010-01-05 6:09 ` Minchan Kim
2010-01-05 6:09 ` KAMEZAWA Hiroyuki
2010-01-05 6:24 ` Minchan Kim
2010-01-05 8:35 ` Peter Zijlstra
2010-01-05 13:45 ` Arjan van de Ven
2010-01-05 14:15 ` Andi Kleen
2010-01-05 15:17 ` Christoph Lameter
2010-01-06 3:22 ` Arjan van de Ven
2010-01-07 16:11 ` Christoph Lameter
2010-01-07 16:19 ` Linus Torvalds
2010-01-07 16:31 ` Linus Torvalds
2010-01-07 16:34 ` Paul E. McKenney
2010-01-07 16:36 ` Christoph Lameter
2010-01-08 4:49 ` Arjan van de Ven
2010-01-08 5:00 ` Linus Torvalds
2010-01-08 15:51 ` Christoph Lameter
2010-01-09 15:55 ` Arjan van de Ven
2010-01-07 17:22 ` Peter Zijlstra
2010-01-07 17:36 ` Linus Torvalds
2010-01-07 17:49 ` Linus Torvalds
2010-01-07 18:00 ` Peter Zijlstra
2010-01-07 18:15 ` Linus Torvalds
2010-01-07 21:49 ` Peter Zijlstra
2010-01-07 18:44 ` Linus Torvalds
2010-01-07 19:20 ` Paul E. McKenney
2010-01-07 20:06 ` Linus Torvalds
2010-01-07 20:25 ` Paul E. McKenney
2010-01-07 19:24 ` Christoph Lameter
2010-01-07 20:08 ` Linus Torvalds
2010-01-07 20:13 ` Linus Torvalds
2010-01-07 21:44 ` Peter Zijlstra
2010-01-07 22:33 ` Linus Torvalds
2010-01-08 0:23 ` KAMEZAWA Hiroyuki
2010-01-08 0:25 ` KAMEZAWA Hiroyuki
2010-01-08 0:39 ` Linus Torvalds
2010-01-08 0:41 ` Linus Torvalds
2010-01-07 23:51 ` Rik van Riel
2010-01-04 18:24 ` [RFC][PATCH 7/8] mm,x86: speculative pagefault support Peter Zijlstra
2010-01-04 18:24 ` [RFC][PATCH 8/8] mm: Optimize pte_map_lock() Peter Zijlstra
2010-01-04 21:41 ` [RFC][PATCH 0/8] Speculative pagefault -v3 Rik van Riel
2010-01-04 21:46 ` Peter Zijlstra
2010-01-04 23:20 ` Rik van Riel
2010-01-04 21:59 ` Christoph Lameter
2010-01-05 0:28 ` KAMEZAWA Hiroyuki
2010-01-05 2:26 ` Minchan Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100106115233.5621bd5e.kamezawa.hiroyu@jp.fujitsu.com \
--to=kamezawa.hiroyu@jp.fujitsu.com \
--cc=a.p.zijlstra@chello.nl \
--cc=cl@linux-foundation.org \
--cc=hugh.dickins@tiscali.co.uk \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan.kim@gmail.com \
--cc=mingo@elte.hu \
--cc=nickpiggin@yahoo.com.au \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).