linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nathan Zimmer <nzimmer@sgi.com>
To: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	adobriyan@gmail.com
Subject: [RFC] Move kfree outside pde_unload_lock
Date: Tue, 21 Aug 2012 15:54:54 -0500	[thread overview]
Message-ID: <20120821205454.GA7772@gulag1.americas.sgi.com> (raw)


I am currently tracking a hotlock reported by a customer on a large, 512 cores,
system, I am currently running 3.6.0 rc1 but the issue looks like it has been
this way for a very long time.
The offending lock is proc_dir_entry->pde_unload_lock.  

In proc_reg_release we are doing a kfree under the spinlock which is ok but it
means we are holding the lock longer then required. Scaling improved when I 
moved kfree out.

Also shouldn't the comment on pde_unload_lock also note that pde_openers and 
pde_unload_completion are both used under the lock?

Here is some data from quick test program which just reads from /proc/cpuinfo.
Lower is better, as you can see the worst case scenario is improved.
	baseline	moved kfree	
tasks	read-sec	read-sec	
1	0.0141		0.0141
2	0.0140		0.0140
4	0.0140		0.0141
8	0.0145		0.0145
16	0.0553		0.0548
32	0.1688		0.1622
64	0.5017		0.3856
128	1.7005		0.9710
256	5.2513		2.6519
512	8.0529		6.2976

If the patch looks agreeable I will resend it properly.
 
diff --git a/fs/proc/inode.c b/fs/proc/inode.c
index 7ac817b..46016c1 100644
--- a/fs/proc/inode.c
+++ b/fs/proc/inode.c
@@ -403,9 +403,11 @@ static int proc_reg_release(struct inode *inode, struct file *file)
 	release = pde->proc_fops->release;
 	if (pdeo) {
 		list_del(&pdeo->lh);
-		kfree(pdeo);
 	}
 	spin_unlock(&pde->pde_unload_lock);
+	if (pdeo) {
+		kfree(pdeo);
+	}
 
 	if (release)
 		rv = release(inode, file);


             reply	other threads:[~2012-08-21 20:54 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-08-21 20:54 Nathan Zimmer [this message]
2012-08-21 22:03 ` [RFC] Move kfree outside pde_unload_lock Alexey Dobriyan
2012-08-22  9:24 ` David Woodhouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20120821205454.GA7772@gulag1.americas.sgi.com \
    --to=nzimmer@sgi.com \
    --cc=adobriyan@gmail.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).