From: Dave Chinner <david@fromorbit.com>
To: Nick Piggin <npiggin@kernel.dk>
Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 00/46] rcu-walk and dcache scaling
Date: Tue, 7 Dec 2010 22:25:55 +1100 [thread overview]
Message-ID: <20101207112555.GE16103@dastard> (raw)
In-Reply-To: <cover.1290852958.git.npiggin@kernel.dk>
On Sat, Nov 27, 2010 at 09:15:58PM +1100, Nick Piggin wrote:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin.git vfs-scale-working
>
> Here is an new set of vfs patches for review, not that there was much interest
> last time they were posted. It is structured like:
>
> * preparation patches
> * introduce new locks to take over dcache_lock, then remove it
> * cleaning up and reworking things for new locks
> * rcu-walk path walking
> * start on some fine grained locking steps
Just got this set of traces doing an 8-way parallel remove of 50
million inodes at about 40M inodes unlinked:
[ 5954.061633] BUG: sleeping function called from invalid context at arch/x86/mm/fault.c:1081
[ 5954.062466] in_atomic(): 0, irqs_disabled(): 1, pid: 2927, name: rm
[ 5954.063122] 3 locks held by rm/2927:
[ 5954.063476] #0: (&sb->s_type->i_mutex_key#12/1){+.+.+.}, at: [<ffffffff8116f5e1>] do_rmdir+0x81/0x130
[ 5954.064014] #1: (&sb->s_type->i_mutex_key#12){+.+.+.}, at: [<ffffffff8116d3a8>] vfs_rmdir+0x58/0xe0
[ 5954.064014] #2: (rcu_read_lock){.+.+..}, at: [<ffffffff811779c0>] shrink_dentry_list+0x0/0x430
[ 5954.064014] irq event stamp: 1484376719
[ 5954.064014] hardirqs last enabled at (1484376719): [<ffffffff810ebf07>] __call_rcu+0xd7/0x1a0
[ 5954.064014] hardirqs last disabled at (1484376718): [<ffffffff810ebe7a>] __call_rcu+0x4a/0x1a0
[ 5954.064014] softirqs last enabled at (1484376586): [<ffffffff8108b911>] __do_softirq+0x161/0x270
[ 5954.064014] softirqs last disabled at (1484376581): [<ffffffff8103af1c>] call_softirq+0x1c/0x50
[ 5954.064014] Pid: 2927, comm: rm Not tainted 2.6.37-rc4-dgc+ #794
[ 5954.064014] Call Trace:
[ 5954.064014] [<ffffffff810b95b0>] ? print_irqtrace_events+0xd0/0xe0
[ 5954.064014] [<ffffffff81076455>] __might_sleep+0xf5/0x130
[ 5954.064014] [<ffffffff81b1e603>] do_page_fault+0x103/0x4f0
[ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
[ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
[ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
[ 5954.064014] [<ffffffff81b19b28>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[ 5954.064014] [<ffffffff810b8e20>] ? usage_match+0x0/0x20
[ 5954.064014] [<ffffffff81b1af25>] page_fault+0x25/0x30
[ 5954.064014] [<ffffffff810b8e20>] ? usage_match+0x0/0x20
[ 5954.064014] [<ffffffff810ba1c8>] ? __bfs+0xc8/0x260
[ 5954.064014] [<ffffffff810ba123>] ? __bfs+0x23/0x260
[ 5954.064014] [<ffffffff810ba4d2>] find_usage_backwards+0x42/0x80
[ 5954.064014] [<ffffffff810bcec4>] check_usage_backwards+0x64/0xf0
[ 5954.064014] [<ffffffff8104796f>] ? save_stack_trace+0x2f/0x50
[ 5954.064014] [<ffffffff810bce60>] ? check_usage_backwards+0x0/0xf0
[ 5954.064014] [<ffffffff810bd9a9>] mark_lock+0x1a9/0x440
[ 5954.064014] [<ffffffff810be989>] __lock_acquire+0x5a9/0x14b0
[ 5954.064014] [<ffffffff810be716>] ? __lock_acquire+0x336/0x14b0
[ 5954.064014] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
[ 5954.064014] [<ffffffff810bf944>] lock_acquire+0xb4/0x140
[ 5954.064014] [<ffffffff81177a1c>] ? shrink_dentry_list+0x5c/0x430
[ 5954.064014] [<ffffffff81b19d86>] _raw_spin_lock+0x36/0x70
[ 5954.064014] [<ffffffff81177a1c>] ? shrink_dentry_list+0x5c/0x430
[ 5954.064014] [<ffffffff81177a1c>] shrink_dentry_list+0x5c/0x430
[ 5954.064014] [<ffffffff811779c0>] ? shrink_dentry_list+0x0/0x430
[ 5954.064014] [<ffffffff816b9c7e>] ? do_raw_spin_unlock+0x5e/0xb0
[ 5954.064014] [<ffffffff81177f2d>] __shrink_dcache_sb+0x13d/0x1c0
[ 5954.064014] [<ffffffff811784bf>] shrink_dcache_parent+0x32f/0x390
[ 5954.064014] [<ffffffff8116d31d>] dentry_unhash+0x3d/0x70
[ 5954.064014] [<ffffffff8116d3b0>] vfs_rmdir+0x60/0xe0
[ 5954.064014] [<ffffffff8116f673>] do_rmdir+0x113/0x130
[ 5954.064014] [<ffffffff8103a03a>] ? sysret_check+0x2e/0x69
[ 5954.064014] [<ffffffff81b19ae9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 5954.064014] [<ffffffff8116f6c5>] sys_unlinkat+0x35/0x40
[ 5954.064014] [<ffffffff8103a002>] system_call_fastpath+0x16/0x1b
[ 5954.092916] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 5954.093806] IP: [<ffffffff810ba1c8>] __bfs+0xc8/0x260
[ 5954.094331] PGD 1084e5067 PUD 102368067 PMD 0
[ 5954.094830] Oops: 0000 [#1] SMP
[ 5954.095194] last sysfs file: /sys/devices/system/cpu/online
[ 5954.095760] CPU 6
[ 5954.095954] Modules linked in:
[ 5954.096319]
[ 5954.096483] Pid: 2927, comm: rm Not tainted 2.6.37-rc4-dgc+ #794 /Bochs
[ 5954.096665] RIP: 0010:[<ffffffff810ba1c8>] [<ffffffff810ba1c8>] __bfs+0xc8/0x260
[ 5954.096665] RSP: 0018:ffff8801175539a8 EFLAGS: 00010046
[ 5954.096665] RAX: ffffffff8267d980 RBX: ffffffff8267d980 RCX: ffff880117553a48
[ 5954.096665] RDX: ffff8801175539d0 RSI: 0000000000000000 RDI: ffff880117553a48
[ 5954.096665] RBP: ffff880117553a08 R08: 0000000000000000 R09: 0000000000000000
[ 5954.096665] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
[ 5954.096665] R13: ffffffff810b8e20 R14: ffff880117553a90 R15: 0000000000000000
[ 5954.096665] FS: 00007f4594cf3700(0000) GS:ffff8800dfa00000(0000) knlGS:0000000000000000
[ 5954.096665] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 5954.096665] CR2: 00007f2f21e89c60 CR3: 0000000110b0f000 CR4: 00000000000006e0
[ 5954.096665] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 5954.096665] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 5954.096665] Process rm (pid: 2927, threadinfo ffff880117552000, task ffff88010ff10b00)
[ 5954.096665] Stack:
[ 5954.096665] ffffffff8267d868 00007fffa02d1428 ffff8800ffffffff ffff880100000000
[ 5954.096665] 000000000000b720 ffff880117553a48 ffffffff8267d868 ffff880117553a48
[ 5954.096665] 0000000000000000 ffff88010ff10b00 0000000000000000 ffffffff81dacba0
[ 5954.096665] Call Trace:
[ 5954.096665] [<ffffffff810ba4d2>] find_usage_backwards+0x42/0x80
[ 5954.096665] [<ffffffff810bcec4>] check_usage_backwards+0x64/0xf0
[ 5954.096665] [<ffffffff8104796f>] ? save_stack_trace+0x2f/0x50
[ 5954.096665] [<ffffffff810bce60>] ? check_usage_backwards+0x0/0xf0
[ 5954.096665] [<ffffffff810bd9a9>] mark_lock+0x1a9/0x440
[ 5954.096665] [<ffffffff810be989>] __lock_acquire+0x5a9/0x14b0
[ 5954.096665] [<ffffffff810be716>] ? __lock_acquire+0x336/0x14b0
[ 5954.096665] [<ffffffff810645b8>] ? pvclock_clocksource_read+0x58/0xd0
[ 5954.096665] [<ffffffff810bf944>] lock_acquire+0xb4/0x140
[ 5954.096665] [<ffffffff81177a1c>] ? shrink_dentry_list+0x5c/0x430
[ 5954.096665] [<ffffffff81b19d86>] _raw_spin_lock+0x36/0x70
[ 5954.096665] [<ffffffff81177a1c>] ? shrink_dentry_list+0x5c/0x430
[ 5954.096665] [<ffffffff81177a1c>] shrink_dentry_list+0x5c/0x430
[ 5954.096665] [<ffffffff811779c0>] ? shrink_dentry_list+0x0/0x430
[ 5954.096665] [<ffffffff816b9c7e>] ? do_raw_spin_unlock+0x5e/0xb0
[ 5954.096665] [<ffffffff81177f2d>] __shrink_dcache_sb+0x13d/0x1c0
[ 5954.096665] [<ffffffff811784bf>] shrink_dcache_parent+0x32f/0x390
[ 5954.096665] [<ffffffff8116d31d>] dentry_unhash+0x3d/0x70
[ 5954.096665] [<ffffffff8116d3b0>] vfs_rmdir+0x60/0xe0
[ 5954.096665] [<ffffffff8116f673>] do_rmdir+0x113/0x130
[ 5954.096665] [<ffffffff8103a03a>] ? sysret_check+0x2e/0x69
[ 5954.096665] [<ffffffff81b19ae9>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 5954.096665] [<ffffffff8116f6c5>] sys_unlinkat+0x35/0x40
[ 5954.096665] [<ffffffff8103a002>] system_call_fastpath+0x16/0x1b
[ 5954.096665] Code: 0a 89 05 dc 0f a8 01 48 8b 41 10 48 85 c0 0f 84 1f 01 00 00 48 8d 98 70 01 00 00 48 05 80 01 00 00 45 85 c0 48 0f 44 d8 4c 8b 3b <49> 8b 07 49 39
[ 5954.096665] RIP [<ffffffff810ba1c8>] __bfs+0xc8/0x260
[ 5954.096665] RSP <ffff8801175539a8>
[ 5954.096665] CR2: 0000000000000000
[ 5954.127991] ---[ end trace 85a6727c2d4e3d90 ]---
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2010-12-07 11:26 UTC|newest]
Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-27 10:15 [PATCH 00/46] rcu-walk and dcache scaling Nick Piggin
2010-11-27 9:44 ` [PATCH 02/46] fs: d_validate fixes Nick Piggin
2010-12-08 1:53 ` Dave Chinner
2010-12-08 6:59 ` Nick Piggin
2010-12-09 0:50 ` Dave Chinner
2010-12-09 0:50 ` Dave Chinner
2010-12-09 4:50 ` Nick Piggin
2010-12-09 4:50 ` Nick Piggin
2010-11-27 9:44 ` [PATCH 03/46] kernel: kmem_ptr_validate considered harmful Nick Piggin
2010-11-27 9:44 ` [PATCH 04/46] fs: dcache documentation cleanup Nick Piggin
2010-11-27 9:44 ` [PATCH 05/46] fs: change d_delete semantics Nick Piggin
2010-11-27 9:44 ` [PATCH 06/46] cifs: dont overwrite dentry name in d_revalidate Nick Piggin
2010-11-27 9:44 ` [PATCH 07/46] jfs: " Nick Piggin
2010-11-27 9:44 ` [PATCH 08/46] fs: change d_compare for rcu-walk Nick Piggin
2010-11-27 9:44 ` [PATCH 09/46] fs: change d_hash " Nick Piggin
2010-11-27 9:44 ` [PATCH 10/46] hostfs: simplify locking Nick Piggin
2010-11-27 9:44 ` [PATCH 11/46] fs: dcache scale hash Nick Piggin
2010-12-09 6:09 ` Dave Chinner
2010-12-09 6:28 ` Nick Piggin
2010-12-09 8:17 ` Dave Chinner
2010-12-09 12:53 ` Nick Piggin
2010-12-09 23:42 ` Dave Chinner
2010-12-10 2:35 ` Nick Piggin
2010-12-10 9:01 ` Dave Chinner
2010-12-13 4:48 ` Nick Piggin
2010-12-13 5:05 ` Nick Piggin
2010-11-27 9:44 ` [PATCH 12/46] fs: dcache scale lru Nick Piggin
2010-12-09 7:22 ` Dave Chinner
2010-12-09 12:34 ` Nick Piggin
2010-11-27 9:44 ` [PATCH 13/46] fs: dcache scale dentry refcount Nick Piggin
2010-11-27 9:44 ` [PATCH 14/46] fs: dcache scale d_unhashed Nick Piggin
2010-11-27 9:44 ` [PATCH 15/46] fs: dcache scale subdirs Nick Piggin
2010-11-27 9:44 ` [PATCH 16/46] fs: scale inode alias list Nick Piggin
2010-11-27 9:44 ` [PATCH 17/46] fs: Use rename lock and RCU for multi-step operations Nick Piggin
2011-01-18 22:32 ` Yehuda Sadeh Weinraub
2011-01-18 22:42 ` Nick Piggin
2011-01-19 22:27 ` Yehuda Sadeh Weinraub
2011-01-19 22:32 ` Nick Piggin
2011-01-25 22:10 ` Yehuda Sadeh Weinraub
2011-01-27 5:18 ` Nick Piggin
2011-02-07 18:52 ` Jim Schutt
2011-02-07 21:04 ` Yehuda Sadeh Weinraub
2011-02-07 21:04 ` Yehuda Sadeh Weinraub
2011-02-07 21:31 ` Jim Schutt
2011-02-07 21:35 ` Gregory Farnum
2011-02-07 22:25 ` Jim Schutt
2011-02-14 17:57 ` Yehuda Sadeh Weinraub
2010-11-27 9:44 ` [PATCH 18/46] fs: increase d_name lock coverage Nick Piggin
2010-11-27 9:44 ` [PATCH 19/46] fs: dcache remove dcache_lock Nick Piggin
2010-11-27 9:44 ` [PATCH 20/46] fs: dcache avoid starvation in dcache multi-step operations Nick Piggin
2010-11-27 9:44 ` [PATCH 21/46] fs: dcache reduce dput locking Nick Piggin
2010-11-27 9:44 ` [PATCH 22/46] fs: dcache reduce locking in d_alloc Nick Piggin
2010-11-27 9:44 ` [PATCH 23/46] fs: dcache reduce dcache_inode_lock Nick Piggin
2010-11-27 9:44 ` [PATCH 24/46] fs: dcache rationalise dget variants Nick Piggin
2010-11-27 9:44 ` [PATCH 25/46] fs: dcache reduce d_parent locking Nick Piggin
2010-11-27 9:44 ` [PATCH 26/46] fs: dcache reduce prune_one_dentry locking Nick Piggin
2010-11-27 9:44 ` [PATCH 27/46] fs: reduce dcache_inode_lock width in lru scanning Nick Piggin
2010-11-27 9:44 ` [PATCH 28/46] fs: use RCU in shrink_dentry_list to reduce lock nesting Nick Piggin
2010-11-27 9:44 ` [PATCH 29/46] fs: consolidate dentry kill sequence Nick Piggin
2010-11-27 9:45 ` [PATCH 30/46] fs: icache RCU free inodes Nick Piggin
2010-11-27 9:45 ` [PATCH 31/46] fs: avoid inode RCU freeing for pseudo fs Nick Piggin
2010-11-27 9:45 ` [PATCH 32/46] kernel: optimise seqlock Nick Piggin
2010-11-27 9:45 ` [PATCH 33/46] fs: rcu-walk for path lookup Nick Piggin
2010-11-27 9:45 ` [PATCH 34/46] fs: fs_struct use seqlock Nick Piggin
2010-11-27 9:45 ` [PATCH 35/46] fs: dcache remove d_mounted Nick Piggin
2010-11-27 9:45 ` [PATCH 36/46] fs: dcache reduce branches in lookup path Nick Piggin
2010-11-27 9:45 ` [PATCH 37/46] fs: cache optimise dentry and inode for rcu-walk Nick Piggin
2010-11-27 9:45 ` [PATCH 38/46] fs: prefetch inode data in dcache lookup Nick Piggin
2010-11-27 9:45 ` [PATCH 39/46] fs: d_revalidate_rcu for rcu-walk Nick Piggin
2010-11-27 9:45 ` [PATCH 40/46] fs: provide rcu-walk aware permission i_ops Nick Piggin
2010-11-27 9:45 ` [PATCH 41/46] fs: provide simple rcu-walk ACL implementation Nick Piggin
2010-11-27 9:45 ` [PATCH 42/46] kernel: add bl_list Nick Piggin
2010-11-27 9:45 ` [PATCH 43/46] bit_spinlock: add required includes Nick Piggin
2010-11-27 9:45 ` [PATCH 44/46] fs: dcache per-bucket dcache hash locking Nick Piggin
2010-11-27 9:45 ` [PATCH 45/46] fs: dcache per-inode inode alias locking Nick Piggin
2010-11-27 9:45 ` [PATCH 46/46] fs: improve scalability of pseudo filesystems Nick Piggin
2010-11-27 9:56 ` [PATCH 01/46] Revert "fs: use RCU read side protection in d_validate" Nick Piggin
2010-12-08 1:16 ` Dave Chinner
2010-12-08 9:38 ` Nick Piggin
2010-12-09 0:44 ` Dave Chinner
2010-12-09 4:38 ` Nick Piggin
2010-12-09 5:16 ` Nick Piggin
2010-11-27 15:04 ` [PATCH 00/46] rcu-walk and dcache scaling Anca Emanuel
2010-11-27 15:04 ` Anca Emanuel
2010-11-28 3:28 ` Nick Piggin
2010-11-28 3:28 ` Nick Piggin
2010-11-28 6:24 ` Sedat Dilek
2010-12-01 18:03 ` David Miller
2010-12-03 16:55 ` Nick Piggin
2010-12-07 11:25 ` Dave Chinner [this message]
2010-12-07 15:24 ` Nick Piggin
2010-12-07 15:24 ` Nick Piggin
2010-12-07 15:49 ` Peter Zijlstra
2010-12-07 15:59 ` Nick Piggin
2010-12-07 16:23 ` Peter Zijlstra
2010-12-08 3:28 ` Nick Piggin
2010-12-07 21:56 ` Dave Chinner
2010-12-08 1:47 ` Nick Piggin
2010-12-08 3:32 ` Dave Chinner
2010-12-08 4:28 ` Dave Chinner
2010-12-08 7:09 ` Nick Piggin
2010-12-08 7:09 ` Nick Piggin
2010-12-10 20:32 ` Paul E. McKenney
2010-12-12 14:54 ` Paul E. McKenney
2010-12-12 14:54 ` Paul E. McKenney
2010-11-27 19:20 Sedat Dilek
2010-11-27 20:53 ` Sedat Dilek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101207112555.GE16103@dastard \
--to=david@fromorbit.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=npiggin@kernel.dk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.