From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751684AbcGNQtu (ORCPT ); Thu, 14 Jul 2016 12:49:50 -0400 Received: from merlin.infradead.org ([205.233.59.134]:46347 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751276AbcGNQts (ORCPT ); Thu, 14 Jul 2016 12:49:48 -0400 Date: Thu, 14 Jul 2016 18:49:45 +0200 From: Peter Zijlstra To: John Stultz Cc: Tejun Heo , Ingo Molnar , lkml , Dmitry Shmidt , Rom Lemarchand , Colin Cross , Todd Kjos , Oleg Nesterov , "Paul E. McKenney" Subject: Re: Severe performance regression w/ 4.4+ on Android due to cgroup locking changes Message-ID: <20160714164945.GH30154@twins.programming.kicks-ass.net> References: <20160713182102.GJ4065@mtj.duckdns.org> <20160713183347.GK4065@mtj.duckdns.org> <20160713201823.GB29670@mtj.duckdns.org> <20160713202657.GW30154@twins.programming.kicks-ass.net> <20160713203944.GC29670@mtj.duckdns.org> <20160713205102.GZ30909@twins.programming.kicks-ass.net> <20160714131809.GO30927@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23.1 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jul 14, 2016 at 09:43:40AM -0700, John Stultz wrote: > On Thu, Jul 14, 2016 at 6:18 AM, Peter Zijlstra wrote: > > On Wed, Jul 13, 2016 at 10:51:02PM +0200, Peter Zijlstra wrote: > >> So, IIRC, the trade-off is a full memory barrier in read_lock and > >> read_unlock() vs sync_sched() in write. > >> > >> Full memory barriers are expensive and while the combined cost might > >> well exceed the cost of the sync_sched() it doesn't suffer the latency > >> issues. > >> > >> Not sure if we can frob the two in a single codebase, but I can have a > >> poke if Oleg or Paul doesn't beat me to it. > > > > OK, not too horrible if I say so myself :-) > > > > The below is a compile tested only first draft so far. I'll go give it > > some runtime next. > > Unfortunately it didn't apply cleanly to the 4.4 based tree I'm > working with, so I had to manually apply the entirety of the > percpu-rwsem.c changes myself. Hopefully I didn't screw it up. > > So running with this, I'm still seeing some pretty large delays. 80ms > peak, with lots of >20ms values as well. > So it doesn't seem to have the positive effect that Paul's change provided. Well that is weird, did you put a tracepoint/printk in synchronize_sched() to ensure we don't end up calling that?