linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Al Viro <viro@ZenIV.linux.org.uk>
To: Waiman Long <waiman.long@hp.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Ingo Molnar <mingo@kernel.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Jeff Layton <jlayton@redhat.com>,
	Miklos Szeredi <mszeredi@suse.cz>, Ingo Molnar <mingo@redhat.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Andi Kleen <andi@firstfloor.org>,
	"Chandramouleeswaran, Aswin" <aswin@hp.com>,
	"Norton, Scott J" <scott.norton@hp.com>
Subject: Re: [PATCH v7 1/4] spinlock: A new lockref structure for lockless update of refcount
Date: Fri, 30 Aug 2013 20:40:59 +0100	[thread overview]
Message-ID: <20130830194059.GC13318@ZenIV.linux.org.uk> (raw)
In-Reply-To: <5220F090.5050908@hp.com>

On Fri, Aug 30, 2013 at 03:20:48PM -0400, Waiman Long wrote:

> There are more contention in the lglock than I remember for the run
> in 3.10. This is an area that I need to look at. In fact, lglock is
> becoming a problem for really large machine with a lot of cores. We
> have a prototype 16-socket machine with 240 cores under development.
> The cost of doing a lg_global_lock will be very high in that type of
> machine given that it is already high in this 80-core machine. I
> have been thinking about instead of per-cpu spinlocks, we could
> change the locking to per-node level. While there will be more
> contention for lg_local_lock, the cost of doing a lg_global_lock
> will be much lower and contention within the local die should not be
> too bad. That will require either a per-node variable infrastructure
> or simulated with the existing per-cpu subsystem.

Speaking of lglock, there's a low-hanging fruit in that area: we have
no reason whatsoever to put anything but regular files with FMODE_WRITE
on the damn per-superblock list - the *only* thing it's used for is
mark_files_ro(), which will skip everything except those.  And since
read opens normally outnumber the writes quite a bit...  Could you
try the diff below and see if it changes the picture?  files_lglock
situation ought to get better...

diff --git a/fs/file_table.c b/fs/file_table.c
index b44e4c5..322cd37 100644
--- a/fs/file_table.c
+++ b/fs/file_table.c
@@ -385,6 +385,10 @@ static inline void __file_sb_list_add(struct file *file, struct super_block *sb)
  */
 void file_sb_list_add(struct file *file, struct super_block *sb)
 {
+	if (likely(!(file->f_mode & FMODE_WRITE)))
+		return;
+	if (!S_ISREG(file_inode(file)->i_mode))
+		return;
 	lg_local_lock(&files_lglock);
 	__file_sb_list_add(file, sb);
 	lg_local_unlock(&files_lglock);
@@ -450,8 +454,6 @@ void mark_files_ro(struct super_block *sb)
 
 	lg_global_lock(&files_lglock);
 	do_file_list_for_each_entry(sb, f) {
-		if (!S_ISREG(file_inode(f)->i_mode))
-		       continue;
 		if (!file_count(f))
 			continue;
 		if (!(f->f_mode & FMODE_WRITE))

  parent reply	other threads:[~2013-08-30 19:40 UTC|newest]

Thread overview: 151+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-06  3:12 [PATCH v7 0/4] Lockless update of reference count protected by spinlock Waiman Long
2013-08-06  3:12 ` [PATCH v7 1/4] spinlock: A new lockref structure for lockless update of refcount Waiman Long
2013-08-29  1:40   ` Linus Torvalds
2013-08-29  4:44     ` Benjamin Herrenschmidt
2013-08-29  7:00       ` Ingo Molnar
2013-08-29 16:43         ` Linus Torvalds
2013-08-29 19:25           ` Linus Torvalds
2013-08-29 23:42             ` Linus Torvalds
2013-08-30  0:26               ` Benjamin Herrenschmidt
2013-08-30  0:49                 ` Linus Torvalds
2013-08-30  2:06                   ` Michael Neuling
2013-08-30  2:30                     ` Benjamin Herrenschmidt
2013-08-30  2:35                       ` Linus Torvalds
2013-08-30  2:45                         ` Benjamin Herrenschmidt
2013-08-30  2:31                     ` Linus Torvalds
2013-08-30  2:43                       ` Benjamin Herrenschmidt
2013-08-30  7:16                   ` Ingo Molnar
2013-08-30 15:28                     ` Linus Torvalds
2013-08-30  3:12               ` Waiman Long
2013-08-30  3:54                 ` Linus Torvalds
2013-08-30  7:55                   ` Sedat Dilek
2013-08-30  8:10                     ` Sedat Dilek
2013-08-30  9:27                     ` Sedat Dilek
2013-08-30  9:48                       ` Ingo Molnar
2013-08-30  9:56                         ` Sedat Dilek
2013-08-30  9:58                           ` Sedat Dilek
2013-08-30 10:29                             ` Sedat Dilek
2013-08-30 10:36                               ` Peter Zijlstra
2013-08-30 10:44                                 ` Sedat Dilek
2013-08-30 10:46                                   ` Sedat Dilek
2013-08-30 10:52                                   ` Peter Zijlstra
2013-08-30 10:57                                     ` Sedat Dilek
2013-08-30 14:05                                       ` Sedat Dilek
2013-08-30 11:19                                 ` Sedat Dilek
2013-08-30 10:38                               ` Sedat Dilek
2013-08-30 15:34                       ` Linus Torvalds
2013-08-30 15:38                         ` Sedat Dilek
2013-08-30 16:12                           ` Steven Rostedt
2013-08-30 16:16                             ` Sedat Dilek
2013-08-30 18:42                             ` Linus Torvalds
2013-08-30 16:32                           ` Linus Torvalds
2013-08-30 16:37                             ` Sedat Dilek
2013-08-30 16:52                               ` Linus Torvalds
2013-08-30 17:11                                 ` Sedat Dilek
2013-08-30 17:26                                   ` Linus Torvalds
2013-09-01 10:01                                 ` Sedat Dilek
2013-09-01 10:33                                   ` Sedat Dilek
2013-09-01 15:32                                   ` Linus Torvalds
2013-09-01 15:45                                     ` Sedat Dilek
2013-09-01 15:55                                       ` Linus Torvalds
2013-09-02 10:30                                         ` Sedat Dilek
2013-09-02 16:09                                           ` David Ahern
2013-09-01 20:59                                     ` Linus Torvalds
2013-09-01 21:23                                       ` Al Viro
2013-09-01 22:16                                         ` Linus Torvalds
2013-09-01 22:35                                           ` Al Viro
2013-09-01 22:44                                             ` Al Viro
2013-09-01 22:58                                               ` Linus Torvalds
2013-09-01 22:48                                           ` Linus Torvalds
2013-09-01 23:30                                             ` Al Viro
2013-09-02  0:12                                               ` Linus Torvalds
2013-09-02  0:50                                                 ` Linus Torvalds
2013-09-02  7:05                                                   ` Ingo Molnar
2013-09-02 16:44                                                     ` Linus Torvalds
2013-09-03 10:15                                                       ` Ingo Molnar
2013-09-03 15:41                                                         ` Linus Torvalds
2013-09-03 18:34                                                           ` Linus Torvalds
2013-09-03 19:19                                                             ` Ingo Molnar
2013-09-03 21:05                                                               ` Linus Torvalds
2013-09-03 21:13                                                                 ` Linus Torvalds
2013-09-03 21:34                                                                   ` Linus Torvalds
2013-09-03 21:39                                                                     ` Linus Torvalds
2013-09-03 14:08                                                       ` Pavel Machek
2013-09-03 22:37                                     ` Sedat Dilek
2013-09-03 22:55                                       ` Dave Jones
2013-09-03 23:05                                         ` Sedat Dilek
2013-09-03 23:15                                           ` Dave Jones
2013-09-03 23:20                                             ` Sedat Dilek
2013-09-03 23:45                                       ` Sedat Dilek
2013-08-30 18:33                   ` Waiman Long
2013-08-30 18:53                     ` Linus Torvalds
2013-08-30 19:20                       ` Waiman Long
2013-08-30 19:33                         ` Linus Torvalds
2013-08-30 20:15                           ` Waiman Long
2013-08-30 20:43                             ` Linus Torvalds
2013-08-30 20:54                               ` Al Viro
2013-08-30 21:03                                 ` Linus Torvalds
2013-08-30 21:44                                   ` Al Viro
2013-08-30 22:30                                     ` Linus Torvalds
2013-08-31 21:23                                       ` Al Viro
2013-08-31 22:49                                         ` Linus Torvalds
2013-08-31 23:27                                           ` Al Viro
2013-09-01  0:13                                             ` Al Viro
2013-09-01 17:48                                               ` Al Viro
2013-09-09  8:30                                               ` Peter Zijlstra
2013-08-30 21:10                                 ` Waiman Long
2013-08-30 21:22                                   ` Linus Torvalds
2013-08-30 21:30                                   ` Al Viro
2013-08-30 21:42                                     ` Waiman Long
2013-08-30 19:40                         ` Al Viro [this message]
2013-08-30 19:52                           ` Waiman Long
2013-08-30 20:26                             ` Al Viro
2013-08-30 20:35                               ` Waiman Long
2013-08-30 20:48                                 ` Al Viro
2013-08-31  2:02                                   ` Waiman Long
2013-08-31  2:35                                     ` Al Viro
2013-08-31  2:42                                       ` Al Viro
2013-09-02 19:25                                         ` Waiman Long
2013-09-03  6:01                                           ` Ingo Molnar
2013-09-03  7:24                                             ` Sedat Dilek
2013-09-03 15:38                                               ` Linus Torvalds
2013-09-03 15:14                                             ` Waiman Long
2013-09-03 15:34                                               ` Linus Torvalds
2013-09-03 19:09                                                 ` Linus Torvalds
2013-09-03 21:01                                                   ` Waiman Long
2013-09-04 14:52                                                   ` Waiman Long
2013-09-04 15:14                                                     ` Linus Torvalds
2013-09-04 19:25                                                       ` Waiman Long
2013-09-04 21:34                                                         ` Linus Torvalds
2013-09-05  2:35                                                           ` Waiman Long
2013-09-05 13:31                                                     ` Ingo Molnar
2013-09-05 17:33                                                       ` Waiman Long
2013-09-05 17:40                                                         ` Ingo Molnar
2013-09-03 22:41                                               ` Sedat Dilek
2013-09-03 23:11                                                 ` Sedat Dilek
2013-09-08 21:45               ` Linus Torvalds
2013-09-09  0:03                 ` Al Viro
2013-09-09  0:25                   ` Linus Torvalds
2013-09-09  0:35                     ` Al Viro
2013-09-09  0:38                       ` Linus Torvalds
2013-09-09  0:57                         ` Al Viro
2013-09-09  2:09                     ` Ramkumar Ramachandra
2013-09-09  0:30                   ` Al Viro
2013-09-09  3:32                   ` Linus Torvalds
2013-09-09  4:06                     ` Ramkumar Ramachandra
2013-09-09  5:44                     ` Al Viro
2013-08-30 17:17           ` Peter Zijlstra
2013-08-30 17:28             ` Linus Torvalds
2013-08-30 17:33               ` Linus Torvalds
2013-08-29 15:20     ` Waiman Long
2013-08-06  3:12 ` [PATCH v7 2/4] spinlock: Enable x86 architecture to do lockless refcount update Waiman Long
2013-08-06  3:12 ` [PATCH v7 3/4] dcache: replace d_lock/d_count by d_lockcnt Waiman Long
2013-08-06  3:12 ` [PATCH v7 4/4] dcache: Enable lockless update of dentry's refcount Waiman Long
2013-08-13 18:03 ` [PATCH v7 0/4] Lockless update of reference count protected by spinlock Waiman Long
2013-08-31  3:06 [PATCH v7 1/4] spinlock: A new lockref structure for lockless update of refcount George Spelvin
2013-08-31 17:16 ` Linus Torvalds
2013-09-01  8:50   ` George Spelvin
2013-09-01 11:10     ` Theodore Ts'o
2013-09-01 15:49       ` Linus Torvalds
2013-09-01 18:11         ` Steven Rostedt
2013-09-01 20:03           ` Linus Torvalds

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130830194059.GC13318@ZenIV.linux.org.uk \
    --to=viro@zeniv.linux.org.uk \
    --cc=andi@firstfloor.org \
    --cc=aswin@hp.com \
    --cc=benh@kernel.crashing.org \
    --cc=jlayton@redhat.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=mingo@redhat.com \
    --cc=mszeredi@suse.cz \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=scott.norton@hp.com \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=waiman.long@hp.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).