From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EBA5C433F5 for ; Tue, 10 May 2022 18:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343802AbiEJSJX (ORCPT ); Tue, 10 May 2022 14:09:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234017AbiEJSJU (ORCPT ); Tue, 10 May 2022 14:09:20 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC550FF1 for ; Tue, 10 May 2022 11:05:21 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id g6so34559041ejw.1 for ; Tue, 10 May 2022 11:05:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uZ5VYfHOn5Pnj/a/F1cEw6h3YGRZnSCs6LxmmCbpo9M=; b=Jbq1zZRw7qvAHeuBtsAOHB+74mcPf8IEkkh12elG1m9IvrvUo1rjeZdIyWRMIlN9/A GWG834bkq0zFSt94OyWBBe70mo3c8KtSK3AEGjrw5gxFpNsr10pq44OvK6poM2jnma5/ iRNbwMU4oQc+Bi0B31Na/m1aYVsTHJk81OOZI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uZ5VYfHOn5Pnj/a/F1cEw6h3YGRZnSCs6LxmmCbpo9M=; b=JNEkeu+WgH4K3F29iM2d/y7qsF5B1Ow3nZR3UTS3Fyw5KEvYZg3eKknAsuMJpVaoNZ UjSkud/y6csqUMI4NBcKnrD27FI0oPcWFPip405IhCrJqWRM08bFDOrT3PLle0WlKgII +bPCtZ+8fj7MHDKQYYVC3VDZAvC9TAdFKx5Op0i4iYFTXxt2tb0cTHrQOpj4Clu1Mt0d WCqoAQjAyrLeNiI+F6xoqXqxA14s5gSLkeb+h4MLQWm/ZIyukje0a+2v4PnrN1frNIAn TDjSX15RKX1Up6L4NI9eR2vwgCNCqBb586/YH8u78MT5l5h/LRvefq7VVGMCo63OHsmN OvtA== X-Gm-Message-State: AOAM532gmo/ySmsYNLOVbbC9EdwGP6QlFLMzDQ/VcrDNldMHN1yIIhyg T2vzqbouOm5ZLuMCa/QN/0ULbe1BCnqo9OjIIP4= X-Google-Smtp-Source: ABdhPJwraQychmLHYGJ7jlqg4NtiVHPox/fuwMC4pUKYYwXx7PmTAAezsy2hnkkSXkgq8+mrN90Ocw== X-Received: by 2002:a17:906:6a0e:b0:6f5:30c9:7c7d with SMTP id qw14-20020a1709066a0e00b006f530c97c7dmr18935660ejc.63.1652205919959; Tue, 10 May 2022 11:05:19 -0700 (PDT) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com. [209.85.221.42]) by smtp.gmail.com with ESMTPSA id b19-20020a170906195300b006f3ef214dbdsm24420eje.35.2022.05.10.11.05.18 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 10 May 2022 11:05:18 -0700 (PDT) Received: by mail-wr1-f42.google.com with SMTP id x18so24950504wrc.0 for ; Tue, 10 May 2022 11:05:18 -0700 (PDT) X-Received: by 2002:adf:dfc8:0:b0:20a:d256:5b5c with SMTP id q8-20020adfdfc8000000b0020ad2565b5cmr19438990wrn.97.1652205918020; Tue, 10 May 2022 11:05:18 -0700 (PDT) MIME-Version: 1.0 References: <20220420013526.GB14333@xsang-OptiPlex-9020> <7d20a9543f69523cfda280e3f5ab17d68db037ab.camel@intel.com> <37dac785a08e3a341bf05d9ee35f19718ce83d26.camel@intel.com> In-Reply-To: <37dac785a08e3a341bf05d9ee35f19718ce83d26.camel@intel.com> From: Linus Torvalds Date: Tue, 10 May 2022 11:05:01 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [mm/page_alloc] f26b3fa046: netperf.Throughput_Mbps -18.0% regression To: "ying.huang@intel.com" , Waiman Long , Peter Zijlstra , Will Deacon Cc: Aaron Lu , Mel Gorman , kernel test robot , Vlastimil Babka , Dave Hansen , Jesper Dangaard Brouer , Michal Hocko , Andrew Morton , LKML , lkp@lists.01.org, kernel test robot , Feng Tang , Zhengjun Xing , fengwei.yin@intel.com Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [ Adding locking people in case they have any input ] On Mon, May 9, 2022 at 11:23 PM ying.huang@intel.com wrote: > > > > > Can you point me to the regression report? I would like to take a look, > > thanks. > > https://lore.kernel.org/all/1425108604.10337.84.camel@linux.intel.com/ Hmm. That explanation looks believable, except that our qspinlocks shouldn't be spinning on the lock itself, but spinning on the mcs node it inserts into the lock. Or so I believed before I looked closer at the code again (it's been years). It turns out we spin on the lock itself if we're the "head waiter". So somebody is always spinning. That's a bit unfortunate for this workload, I guess. I think from a pure lock standpoint, it's the right thing to do (no unnecessary bouncing, with the lock releaser doing just one write, and the head waiter spinning on it is doing the right thing). But I think this is an example of where you end up having that spinning on the lock possibly then being a disturbance on the other fields around the lock. I wonder if Waiman / PeterZ / Will have any comments on that. Maybe that "spin on the lock itself" is just fundamentally the only correct thing, but since my initial reaction was "no, we're spinning on the mcs node", maybe that would be _possible_? We do have a lot of those spinlocks embedded in other data structures cases. And if "somebody else is waiting for the lock" contends badly with "the lock holder is doing a lot of writes close to the lock", then that's not great. Linus