From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753193AbcHPApV (ORCPT ); Mon, 15 Aug 2016 20:45:21 -0400 Received: from mail-oi0-f66.google.com ([209.85.218.66]:34959 "EHLO mail-oi0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750796AbcHPApU (ORCPT ); Mon, 15 Aug 2016 20:45:20 -0400 MIME-Version: 1.0 In-Reply-To: <20160816001721.GC19025@dastard> References: <20160815004826.GW19025@dastard> <20160815022808.GX19025@dastard> <20160815050016.GY19025@dastard> <20160815222211.GA19025@dastard> <20160816001721.GC19025@dastard> From: Linus Torvalds Date: Mon, 15 Aug 2016 17:45:17 -0700 X-Google-Sender-Auth: mDO_AOX92UvNjwAY7VhV4UFW4zg Message-ID: Subject: Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression To: Dave Chinner Cc: Bob Peterson , "Kirill A. Shutemov" , "Huang, Ying" , Christoph Hellwig , Wu Fengguang , LKP , Tejun Heo , LKML Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 15, 2016 at 5:17 PM, Dave Chinner wrote: > > Read the code, Linus? I am. It's how I came up with my current pet theory. But I don't actually have enough sane numbers to make it much more than a cute pet theory. It *might* explain why you see tons of kswap time and bad lock contention where it didn't use to exist, but .. I can't recreate the problem, and your old profiles were bad enough that they aren't really worth looking at. > Except they *aren't broken*. They are simply *less accurate* than > they could be. They are so much less accurate that quite frankly, there's no point in looking at them outside of "there is contention on the lock". And considering that the numbers didn't even change when you had spinlock debugging on, it's not the lock itself that causes this, I'm pretty sure. Because when you have normal contention due to the *locking* itself being the problem, it tends to absolutely _explode_ with the debugging spinlocks, because the lock itself becomes much more expensive. Usually super-linearly. But that wasn't the case here. The numbers stayed constant. So yeah, I started looking at bigger behavioral issues, which is why I zeroed in on that zone-vs-node change. But it might be a completely broken theory. For example, if you still have the contention when running plain 4.7, that theory was clearly complete BS. And this is where "less accurate" means that they are almost entirely useless. More detail needed. It might not be in the profiles themselves, of course. There might be other much more informative sources if you can come up with anything... Linus From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============1524670144010122437==" MIME-Version: 1.0 From: Linus Torvalds To: lkp@lists.01.org Subject: Re: [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression Date: Mon, 15 Aug 2016 17:45:17 -0700 Message-ID: In-Reply-To: <20160816001721.GC19025@dastard> List-Id: --===============1524670144010122437== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Mon, Aug 15, 2016 at 5:17 PM, Dave Chinner wrote: > > Read the code, Linus? I am. It's how I came up with my current pet theory. But I don't actually have enough sane numbers to make it much more than a cute pet theory. It *might* explain why you see tons of kswap time and bad lock contention where it didn't use to exist, but .. I can't recreate the problem, and your old profiles were bad enough that they aren't really worth looking at. > Except they *aren't broken*. They are simply *less accurate* than > they could be. They are so much less accurate that quite frankly, there's no point in looking at them outside of "there is contention on the lock". And considering that the numbers didn't even change when you had spinlock debugging on, it's not the lock itself that causes this, I'm pretty sure. Because when you have normal contention due to the *locking* itself being the problem, it tends to absolutely _explode_ with the debugging spinlocks, because the lock itself becomes much more expensive. Usually super-linearly. But that wasn't the case here. The numbers stayed constant. So yeah, I started looking at bigger behavioral issues, which is why I zeroed in on that zone-vs-node change. But it might be a completely broken theory. For example, if you still have the contention when running plain 4.7, that theory was clearly complete BS. And this is where "less accurate" means that they are almost entirely usele= ss. More detail needed. It might not be in the profiles themselves, of course. There might be other much more informative sources if you can come up with anything... Linus --===============1524670144010122437==--