From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CCA4C433E1 for ; Fri, 21 Aug 2020 08:55:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 39A63207DF for ; Fri, 21 Aug 2020 08:55:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39A63207DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=Huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CA8848D0030; Fri, 21 Aug 2020 04:55:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C58E48D0005; Fri, 21 Aug 2020 04:55:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B6F618D0030; Fri, 21 Aug 2020 04:55:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0091.hostedemail.com [216.40.44.91]) by kanga.kvack.org (Postfix) with ESMTP id A1FFE8D0005 for ; Fri, 21 Aug 2020 04:55:51 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6EBBC824556B for ; Fri, 21 Aug 2020 08:55:51 +0000 (UTC) X-FDA: 77173968102.29.grape80_080270727038 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 3C33818086E48 for ; Fri, 21 Aug 2020 08:55:51 +0000 (UTC) X-HE-Tag: grape80_080270727038 X-Filterd-Recvd-Size: 5684 Received: from huawei.com (lhrrgout.huawei.com [185.176.76.210]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Fri, 21 Aug 2020 08:55:50 +0000 (UTC) Received: from lhreml710-chm.china.huawei.com (unknown [172.18.7.107]) by Forcepoint Email with ESMTP id E386C4B18AC1B04972AA; Fri, 21 Aug 2020 09:55:48 +0100 (IST) Received: from localhost (10.52.123.86) by lhreml710-chm.china.huawei.com (10.201.108.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1913.5; Fri, 21 Aug 2020 09:55:47 +0100 Date: Fri, 21 Aug 2020 09:54:14 +0100 From: Jonathan Cameron To: Bjorn Helgaas CC: , , , , Lorenzo Pieralisi , Bjorn Helgaas , , , Ingo Molnar , Thomas Gleixner , , Dan Williams , "Brice Goglin" , Sean V Kelley , Subject: Re: [PATCH v9 2/6] x86: Support Generic Initiator only proximity domains Message-ID: <20200821095414.00004559@Huawei.com> In-Reply-To: <20200820222433.GA1571517@bjorn-Precision-5520> References: <20200819145111.1715026-3-Jonathan.Cameron@huawei.com> <20200820222433.GA1571517@bjorn-Precision-5520> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; i686-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.52.123.86] X-ClientProxiedBy: lhreml701-chm.china.huawei.com (10.201.108.50) To lhreml710-chm.china.huawei.com (10.201.108.61) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 3C33818086E48 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 20 Aug 2020 17:24:33 -0500 Bjorn Helgaas wrote: > On Wed, Aug 19, 2020 at 10:51:07PM +0800, Jonathan Cameron wrote: > > In common with memoryless domains we only register GI domains > > if the proximity node is not online. If a domain is already > > a memory containing domain, or a memoryless domain there is > > nothing to do just because it also contains a Generic Initiator. > > > > Signed-off-by: Jonathan Cameron > > --- > > arch/x86/include/asm/numa.h | 2 ++ > > arch/x86/kernel/setup.c | 1 + > > arch/x86/mm/numa.c | 14 ++++++++++++++ > > 3 files changed, 17 insertions(+) > > > > diff --git a/arch/x86/include/asm/numa.h b/arch/x86/include/asm/numa.h > > index bbfde3d2662f..f631467272a3 100644 > > --- a/arch/x86/include/asm/numa.h > > +++ b/arch/x86/include/asm/numa.h > > @@ -62,12 +62,14 @@ extern void numa_clear_node(int cpu); > > extern void __init init_cpu_to_node(void); > > extern void numa_add_cpu(int cpu); > > extern void numa_remove_cpu(int cpu); > > +extern void init_gi_nodes(void); > > #else /* CONFIG_NUMA */ > > static inline void numa_set_node(int cpu, int node) { } > > static inline void numa_clear_node(int cpu) { } > > static inline void init_cpu_to_node(void) { } > > static inline void numa_add_cpu(int cpu) { } > > static inline void numa_remove_cpu(int cpu) { } > > +static inline void init_gi_nodes(void) { } > > #endif /* CONFIG_NUMA */ > > > > #ifdef CONFIG_DEBUG_PER_CPU_MAPS > > diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c > > index 3511736fbc74..9062c146f03a 100644 > > --- a/arch/x86/kernel/setup.c > > +++ b/arch/x86/kernel/setup.c > > @@ -1218,6 +1218,7 @@ void __init setup_arch(char **cmdline_p) > > prefill_possible_map(); > > > > init_cpu_to_node(); > > + init_gi_nodes(); > > > > io_apic_init_mappings(); > > > > diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c > > index aa76ec2d359b..fc630dc6764e 100644 > > --- a/arch/x86/mm/numa.c > > +++ b/arch/x86/mm/numa.c > > @@ -747,6 +747,20 @@ static void __init init_memory_less_node(int nid) > > */ > > } > > > > +/* > > + * Generic Initiator Nodes may have neither CPU nor Memory. > > + * At this stage if either of the others were present we would > > + * already be online. > > + */ > > +void __init init_gi_nodes(void) > > +{ > > + int nid; > > + > > + for_each_node_state(nid, N_GENERIC_INITIATOR) > > + if (!node_online(nid)) > > + init_memory_less_node(nid); > > +} > > This doesn't *look* very x86-specific, and apparently you don't need > any arm64-specific changes? Too bad this can't be unified a little > bit to remove the arch #ifdefs completely. The only ifdefs in the set aren't actually about this (which is entirely contained in arch code) they are to avoid ia64 issues as has an entirely different implementation that I doubt anyone wants to touch! > > I do see that init_memory_less_node() is only implemented on x86, but > it just seems like all this might not be inherently be arch-specific. I absolutely agree with a long term aim to unify the numa setup code across architectures. The x86 code is rather more involved than what we have for arm64. On arm64, memoryless nodes are handled in the same pass as those with memory (and we get GI nodes for free). There are some separate patches under discussion that take a few steps in that direction. https://lore.kernel.org/linux-arm-kernel/20200814214725.28818-3-atish.patra@wdc.com/T/ https://patchwork.kernel.org/patch/11651437/ (this one is mostly by coincidence rather than intent!) Jonathan