From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3871AC433ED for ; Fri, 30 Apr 2021 01:48:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E6F4B613D8 for ; Fri, 30 Apr 2021 01:48:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229884AbhD3Bsw (ORCPT ); Thu, 29 Apr 2021 21:48:52 -0400 Received: from mga18.intel.com ([134.134.136.126]:1029 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229577AbhD3Bsv (ORCPT ); Thu, 29 Apr 2021 21:48:51 -0400 IronPort-SDR: X+g1ejYcjJ2JXyZvSASSPzFXNyvYYPhXEXBz9x8QV5D72pr0XxNh6y+g5rabSC3g2hZk42E2uE ADw0yXK1IwTw== X-IronPort-AV: E=McAfee;i="6200,9189,9969"; a="184646243" X-IronPort-AV: E=Sophos;i="5.82,260,1613462400"; d="scan'208";a="184646243" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2021 18:48:03 -0700 IronPort-SDR: VCNCi44upo9HW3I6bWsf05cfpZFM8Nfapyq8zjSxUuparGWrXoSKUUhB0YE+Tzb9aHx5t92MMt U2O1V25OFqRg== X-IronPort-AV: E=Sophos;i="5.82,260,1613462400"; d="scan'208";a="431182560" Received: from jbrandeb-mobl4.amr.corp.intel.com (HELO localhost) ([10.209.109.170]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2021 18:48:03 -0700 Date: Thu, 29 Apr 2021 18:48:02 -0700 From: Jesse Brandeburg To: Nitesh Lal Cc: Thomas Gleixner , "frederic@kernel.org" , "juri.lelli@redhat.com" , Marcelo Tosatti , abelits@marvell.com, Robin Murphy , "linux-kernel@vger.kernel.org" , "linux-api@vger.kernel.org" , "bhelgaas@google.com" , "linux-pci@vger.kernel.org" , "rostedt@goodmis.org" , "mingo@kernel.org" , "peterz@infradead.org" , "davem@davemloft.net" , "akpm@linux-foundation.org" , "sfr@canb.auug.org.au" , "stephen@networkplumber.org" , "rppt@linux.vnet.ibm.com" , "jinyuqi@huawei.com" , "zhangshaokun@hisilicon.com" , netdev@vger.kernel.org, chris.friesen@windriver.com Subject: Re: [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs Message-ID: <20210429184802.0000641e@intel.com> In-Reply-To: References: <20200625223443.2684-1-nitesh@redhat.com> <20200625223443.2684-2-nitesh@redhat.com> <3e9ce666-c9cd-391b-52b6-3471fe2be2e6@arm.com> <20210127121939.GA54725@fuller.cnet> <87r1m5can2.fsf@nanos.tec.linutronix.de> <20210128165903.GB38339@fuller.cnet> <87h7n0de5a.fsf@nanos.tec.linutronix.de> <20210204181546.GA30113@fuller.cnet> <20210204190647.GA32868@fuller.cnet> <87y2g26tnt.fsf@nanos.tec.linutronix.de> <7780ae60-efbd-2902-caaa-0249a1f277d9@redhat.com> <07c04bc7-27f0-9c07-9f9e-2d1a450714ef@redhat.com> <20210406102207.0000485c@intel.com> <1a044a14-0884-eedb-5d30-28b4bec24b23@redhat.com> <20210414091100.000033cf@intel.com> <54ecc470-b205-ea86-1fc3-849c5b144b3b@redhat.com> X-Mailer: Claws Mail 3.12.0 (GTK+ 2.24.28; i686-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Nitesh Lal wrote: > @Jesse do you think the Part-1 findings explain the behavior that you have > observed in the past? > > Also, let me know if there are any suggestions or experiments to try here. Wow Nitesh, nice work! That's quite a bit of spelunking you had to do there! Your results that show the older kernels with ranged affinity issues is consistent with what I remember from that time, and the original problem. I'm glad to see that a) Thomas fixed the kernel to even do better than ranged affinity masks, and that b) if you revert my patch, the new behavior is better and still maintains the fix from a). For me this explains the whole picture and makes me feel comfortable with the patch that reverts the initial affinity mask (that also introduces a subtle bug with the reserved CPUs that I believe you've noted already). Thanks for this work! Jesse