From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1755037AbbFWTgk (ORCPT <rfc822;w@1wt.eu>);
	Tue, 23 Jun 2015 15:36:40 -0400
Received: from casper.infradead.org ([85.118.1.10]:60028 "EHLO
	casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751431AbbFWTgd (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Tue, 23 Jun 2015 15:36:33 -0400
Date: Tue, 23 Jun 2015 21:36:24 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Daniel Wagner <daniel.wagner@bmw-carit.de>
Cc: oleg@redhat.com, paulmck@linux.vnet.ibm.com, tj@kernel.org,
        mingo@redhat.com, linux-kernel@vger.kernel.org, der.herr@hofr.at,
        dave@stgolabs.net, riel@redhat.com, viro@ZenIV.linux.org.uk,
        torvalds@linux-foundation.org, jlayton@poochiereds.net
Subject: Re: [RFC][PATCH 00/13] percpu rwsem -v2
Message-ID: <20150623193624.GH18673@twins.programming.kicks-ass.net>
References: <20150622121623.291363374@infradead.org>
 <55884FC2.6030607@bmw-carit.de>
 <20150622190553.GZ3644@twins.programming.kicks-ass.net>
 <5589285C.2010100@bmw-carit.de>
 <20150623143411.GA25159@twins.programming.kicks-ass.net>
 <558973A7.6010407@bmw-carit.de>
 <20150623175012.GD3644@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150623175012.GD3644@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Jun 23, 2015 at 07:50:12PM +0200, Peter Zijlstra wrote:
> On Tue, Jun 23, 2015 at 04:56:39PM +0200, Daniel Wagner wrote:
> > flock02
> >                              mean   variance      sigma        max        min
> >                     tip-1    11.8994     0.5874     0.7664    13.2022     8.6324
> >                     tip-2    11.7394     0.5252     0.7247    13.2540     9.7513
> >                     tip-3    11.8155     0.5288     0.7272    13.2700     9.9480
> >        tip+percpu-rswem-1    15.3601     0.8981     0.9477    16.8116    12.6910
> >        tip+percpu-rswem-2    15.2558     0.8442     0.9188    17.0199    12.9586
> >        tip+percpu-rswem-3    15.5297     0.6386     0.7991    17.4392    12.7992
> 
> I did indeed manage to get flock02 down to a usable level and found:

Aside from the flock_lock_file function moving up, we also get an
increase in _raw_spin_lock.

Before:

     5.17%     5.17%  flock02       [kernel.vmlinux]            [k] _raw_spin_lock
                 |
                 ---_raw_spin_lock
                    |          
                    |--99.75%-- flock_lock_file_wait
                    |          sys_flock
                    |          entry_SYSCALL_64_fastpath
                    |          flock
                     --0.25%-- [...]


After:

     7.20%     7.20%  flock02       [kernel.vmlinux]            [k] _raw_spin_lock
                 |
                 ---_raw_spin_lock
                    |          
                    |--52.23%-- flock_lock_file_wait
                    |          sys_flock
                    |          entry_SYSCALL_64_fastpath
                    |          flock
                    |          
                    |--25.92%-- flock_lock_file
                    |          flock_lock_file_wait
                    |          sys_flock
                    |          entry_SYSCALL_64_fastpath
                    |          flock
                    |          
                    |--21.42%-- locks_delete_lock_ctx
                    |          flock_lock_file
                    |          flock_lock_file_wait
                    |          sys_flock
                    |          entry_SYSCALL_64_fastpath
                    |          flock
                     --0.43%-- [...]


And its not at all clear to me why this would be. It looks like
FILE_LOCK_DEFERRED is happening, but I've not yet figured out why that
would be.