From mboxrd@z Thu Jan 1 00:00:00 1970 From: Davidlohr Bueso Subject: Re: [RFC] Heterogeneous memory management (mirror process address space on a device mmu). Date: Tue, 06 May 2014 19:33:07 -0700 Message-ID: <1399429987.2581.25.camel@buesod1.americas.hpqcorp.net> References: <1399038730-25641-1-git-send-email-j.glisse@gmail.com> <20140506102925.GD11096@twins.programming.kicks-ass.net> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: j.glisse@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mel Gorman , "H. Peter Anvin" , Andrew Morton , Linda Wang , Kevin E Martin , Jerome Glisse , Andrea Arcangeli , Johannes Weiner , Larry Woodman , Rik van Riel , Dave Airlie , Jeff Law , Brendan Conoboy , Joe Donohue , Duncan Poole , Sherry Cheung , Subhash Gutti , John Hubbard , Mark Hairgrove , Lucien Dunning , Cameron Buschardt , Arvind Gopalakrishnan Return-path: In-Reply-To: <20140506102925.GD11096@twins.programming.kicks-ass.net> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Tue, 2014-05-06 at 12:29 +0200, Peter Zijlstra wrote: > So you forgot to CC Linus, Linus has expressed some dislike for > preemptible mmu_notifiers in the recent past: > > https://lkml.org/lkml/2013/9/30/385 I'm glad this came up again. So I've been running benchmarks (mostly aim7, which nicely exercises our locks) comparing my recent v4 for rwsem optimistic spinning against previous implementation ideas for the anon-vma lock, mostly: - rwsem (currently) - rwlock_t - qrwlock_t - rwsem+optspin Of course, *any* change provides significant improvement in throughput for several workloads, by avoiding to block -- there are more performance numbers in the different patches. This is fairly obvious. What is perhaps not so obvious is that rwsem+optimistic spinning beats all others, including the improved qrwlock from Waiman and Peter. This is mostly because of the idea of cancelable MCS, which was mimic'ed from mutexes. The delta in most cases is around +10-15%, which is non trivial. I mention this because from a performance PoV, we'll stop caring so much about the type of lock we require in the notifier related code. So while this is not conclusive, I'm not as opposed to keeping the locks blocking as I once was. Now this might still imply things like poor design choices, but that's neither here nor there. /me sees Sagi smiling ;) Thanks, Davidlohr -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f178.google.com (mail-ob0-f178.google.com [209.85.214.178]) by kanga.kvack.org (Postfix) with ESMTP id D6AF46B0035 for ; Tue, 6 May 2014 22:33:17 -0400 (EDT) Received: by mail-ob0-f178.google.com with SMTP id va2so430026obc.23 for ; Tue, 06 May 2014 19:33:17 -0700 (PDT) Received: from g5t1625.atlanta.hp.com (g5t1625.atlanta.hp.com. [15.192.137.8]) by mx.google.com with ESMTPS id gu8si9719742obc.139.2014.05.06.19.33.16 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 06 May 2014 19:33:16 -0700 (PDT) Message-ID: <1399429987.2581.25.camel@buesod1.americas.hpqcorp.net> Subject: Re: [RFC] Heterogeneous memory management (mirror process address space on a device mmu). From: Davidlohr Bueso Date: Tue, 06 May 2014 19:33:07 -0700 In-Reply-To: <20140506102925.GD11096@twins.programming.kicks-ass.net> References: <1399038730-25641-1-git-send-email-j.glisse@gmail.com> <20140506102925.GD11096@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Peter Zijlstra Cc: j.glisse@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mel Gorman , "H. Peter Anvin" , Andrew Morton , Linda Wang , Kevin E Martin , Jerome Glisse , Andrea Arcangeli , Johannes Weiner , Larry Woodman , Rik van Riel , Dave Airlie , Jeff Law , Brendan Conoboy , Joe Donohue , Duncan Poole , Sherry Cheung , Subhash Gutti , John Hubbard , Mark Hairgrove , Lucien Dunning , Cameron Buschardt , Arvind Gopalakrishnan , Haggai Eran , Or Gerlitz , Sagi Grimberg , Shachar Raindel , Liran Liss , Roland Dreier , "Sander, Ben" , "Stoner, Greg" , "Bridgman, John" , "Mantor, Michael" , "Blinzer, Paul" , "Morichetti, Laurent" , "Deucher, Alexander" , "Gabbay, Oded" , Linus Torvalds On Tue, 2014-05-06 at 12:29 +0200, Peter Zijlstra wrote: > So you forgot to CC Linus, Linus has expressed some dislike for > preemptible mmu_notifiers in the recent past: > > https://lkml.org/lkml/2013/9/30/385 I'm glad this came up again. So I've been running benchmarks (mostly aim7, which nicely exercises our locks) comparing my recent v4 for rwsem optimistic spinning against previous implementation ideas for the anon-vma lock, mostly: - rwsem (currently) - rwlock_t - qrwlock_t - rwsem+optspin Of course, *any* change provides significant improvement in throughput for several workloads, by avoiding to block -- there are more performance numbers in the different patches. This is fairly obvious. What is perhaps not so obvious is that rwsem+optimistic spinning beats all others, including the improved qrwlock from Waiman and Peter. This is mostly because of the idea of cancelable MCS, which was mimic'ed from mutexes. The delta in most cases is around +10-15%, which is non trivial. I mention this because from a performance PoV, we'll stop caring so much about the type of lock we require in the notifier related code. So while this is not conclusive, I'm not as opposed to keeping the locks blocking as I once was. Now this might still imply things like poor design choices, but that's neither here nor there. /me sees Sagi smiling ;) Thanks, Davidlohr -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org