From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yunsheng Lin Date: Tue, 08 Oct 2019 08:38:28 +0000 Subject: Re: [PATCH v6] numa: make node_to_cpumask_map() NUMA_NO_NODE aware Message-Id: <47fa4cee-8528-7c23-c7de-7be1b65aa2ae@huawei.com> List-Id: References: <20190924091714.GJ2369@hirez.programming.kicks-ass.net> <20190924105622.GH23050@dhcp22.suse.cz> <20190924112349.GJ2332@hirez.programming.kicks-ass.net> <20190924115401.GM23050@dhcp22.suse.cz> <20190924120943.GP2349@hirez.programming.kicks-ass.net> <20190924122500.GP23050@dhcp22.suse.cz> <20190924124325.GQ2349@hirez.programming.kicks-ass.net> <20190924125936.GR2349@hirez.programming.kicks-ass.net> <20190924131939.GS23050@dhcp22.suse.cz> <1adcbe68-6753-3497-48a0-cc84ac503372@huawei.com> <20190925104108.GE4553@hirez.programming.kicks-ass.net> In-Reply-To: <20190925104108.GE4553@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Peter Zijlstra Cc: Michal Hocko , catalin.marinas@arm.com, will@kernel.org, mingo@redhat.com, bp@alien8.de, rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, heiko.carstens@de.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, ysato@users.sourceforge.jp, dalias@libc.org, davem@davemloft.net, ralf@linux-mips.org, paul.burton@mips.com, jhogan@kernel.org, jiaxun.yang@flygoat.com, chenhc@lemote.com, akpm@linux-foundation.org, rppt@linux.ibm.com, anshuman.khandual@arm.com, tglx@linutronix.de, cai@lca.pw, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, hpa@zytor.com, x86@kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, len.brown@intel.com, axboe@kernel.dk, dledford@redhat.com, jeffrey.t.kirsher@intel.com, linux-alpha@vger.kernel.org, naveen.n.rao@linux.vnet.ibm.com, mwb@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, tbogendoerfer@suse.de, linux-mips@vger.kernel.org, rafael@kernel.org, gregkh@linuxfoundation.org On 2019/9/25 18:41, Peter Zijlstra wrote: > On Wed, Sep 25, 2019 at 05:14:20PM +0800, Yunsheng Lin wrote: >> From the discussion above, It seems making the node_to_cpumask_map() >> NUMA_NO_NODE aware is the most feasible way to move forwad. > > That's still wrong. Hi, Peter It seems this has trapped in the dead circle. >From my understanding, NUMA_NO_NODE which means not node numa preference is the state to describe the node of virtual device or the physical device that has equal distance to all cpu. We can be stricter if the device does have a nearer node, but we can not deny that a device does not have a node numa preference or node affinity, which also means the control or data buffer can be allocated at the node where the process is running. As you has proposed, making it -2 and have dev_to_node() warn if the device does have a nearer node and not set by the fw is a way to be stricter. But I think maybe being stricter is not really relevant to NUMA_NO_NODE, because we does need a state to describe the device that have equal distance to all node, even if it is not physically scalable. Any better suggestion to move this forward? > > . > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EE22C4360C for ; Tue, 8 Oct 2019 08:38:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0ECDC20673 for ; Tue, 8 Oct 2019 08:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730183AbfJHIim (ORCPT ); Tue, 8 Oct 2019 04:38:42 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:41892 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730063AbfJHIil (ORCPT ); Tue, 8 Oct 2019 04:38:41 -0400 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 52ADD745F3E9E7BF8522; Tue, 8 Oct 2019 16:38:39 +0800 (CST) Received: from [127.0.0.1] (10.74.191.121) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.439.0; Tue, 8 Oct 2019 16:38:37 +0800 Subject: Re: [PATCH v6] numa: make node_to_cpumask_map() NUMA_NO_NODE aware To: Peter Zijlstra CC: Michal Hocko , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20190924091714.GJ2369@hirez.programming.kicks-ass.net> <20190924105622.GH23050@dhcp22.suse.cz> <20190924112349.GJ2332@hirez.programming.kicks-ass.net> <20190924115401.GM23050@dhcp22.suse.cz> <20190924120943.GP2349@hirez.programming.kicks-ass.net> <20190924122500.GP23050@dhcp22.suse.cz> <20190924124325.GQ2349@hirez.programming.kicks-ass.net> <20190924125936.GR2349@hirez.programming.kicks-ass.net> <20190924131939.GS23050@dhcp22.suse.cz> <1adcbe68-6753-3497-48a0-cc84ac503372@huawei.com> <20190925104108.GE4553@hirez.programming.kicks-ass.net> From: Yunsheng Lin Message-ID: <47fa4cee-8528-7c23-c7de-7be1b65aa2ae@huawei.com> Date: Tue, 8 Oct 2019 16:38:28 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20190925104108.GE4553@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.74.191.121] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2019/9/25 18:41, Peter Zijlstra wrote: > On Wed, Sep 25, 2019 at 05:14:20PM +0800, Yunsheng Lin wrote: >> From the discussion above, It seems making the node_to_cpumask_map() >> NUMA_NO_NODE aware is the most feasible way to move forwad. > > That's still wrong. Hi, Peter It seems this has trapped in the dead circle. >From my understanding, NUMA_NO_NODE which means not node numa preference is the state to describe the node of virtual device or the physical device that has equal distance to all cpu. We can be stricter if the device does have a nearer node, but we can not deny that a device does not have a node numa preference or node affinity, which also means the control or data buffer can be allocated at the node where the process is running. As you has proposed, making it -2 and have dev_to_node() warn if the device does have a nearer node and not set by the fw is a way to be stricter. But I think maybe being stricter is not really relevant to NUMA_NO_NODE, because we does need a state to describe the device that have equal distance to all node, even if it is not physically scalable. Any better suggestion to move this forward? > > . > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: Re: [PATCH v6] numa: make node_to_cpumask_map() NUMA_NO_NODE aware References: <20190924091714.GJ2369@hirez.programming.kicks-ass.net> <20190924105622.GH23050@dhcp22.suse.cz> <20190924112349.GJ2332@hirez.programming.kicks-ass.net> <20190924115401.GM23050@dhcp22.suse.cz> <20190924120943.GP2349@hirez.programming.kicks-ass.net> <20190924122500.GP23050@dhcp22.suse.cz> <20190924124325.GQ2349@hirez.programming.kicks-ass.net> <20190924125936.GR2349@hirez.programming.kicks-ass.net> <20190924131939.GS23050@dhcp22.suse.cz> <1adcbe68-6753-3497-48a0-cc84ac503372@huawei.com> <20190925104108.GE4553@hirez.programming.kicks-ass.net> From: Yunsheng Lin Message-ID: <47fa4cee-8528-7c23-c7de-7be1b65aa2ae@huawei.com> Date: Tue, 8 Oct 2019 16:38:28 +0800 MIME-Version: 1.0 In-Reply-To: <20190925104108.GE4553@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: To: Peter Zijlstra Cc: Michal Hocko , catalin.marinas@arm.com, will@kernel.org, mingo@redhat.com, bp@alien8.de, rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, heiko.carstens@de.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, ysato@users.sourceforge.jp, dalias@libc.org, davem@davemloft.net, ralf@linux-mips.org, paul.burton@mips.com, jhogan@kernel.org, jiaxun.yang@flygoat.com, chenhc@lemote.com, akpm@linux-foundation.org, rppt@linux.ibm.com, anshuman.khandual@arm.com, tglx@linutronix.de, cai@lca.pw, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, hpa@zytor.com, x86@kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, len.brown@intel.com, axboe@kernel.dk, dledford@redhat.com, jeffrey.t.kirsher@intel.com, linux-alpha@vger.kernel.org, naveen.n.rao@linux.vnet.ibm.com, mwb@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, tbogendoerfer@suse.de, linux-mips@vger.kernel.org, rafael@kernel.org, gregkh@linuxfoundation.org On 2019/9/25 18:41, Peter Zijlstra wrote: > On Wed, Sep 25, 2019 at 05:14:20PM +0800, Yunsheng Lin wrote: >> From the discussion above, It seems making the node_to_cpumask_map() >> NUMA_NO_NODE aware is the most feasible way to move forwad. > > That's still wrong. Hi, Peter It seems this has trapped in the dead circle. >From my understanding, NUMA_NO_NODE which means not node numa preference is the state to describe the node of virtual device or the physical device that has equal distance to all cpu. We can be stricter if the device does have a nearer node, but we can not deny that a device does not have a node numa preference or node affinity, which also means the control or data buffer can be allocated at the node where the process is running. As you has proposed, making it -2 and have dev_to_node() warn if the device does have a nearer node and not set by the fw is a way to be stricter. But I think maybe being stricter is not really relevant to NUMA_NO_NODE, because we does need a state to describe the device that have equal distance to all node, even if it is not physically scalable. Any better suggestion to move this forward? > > . > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F0D9C4360C for ; Tue, 8 Oct 2019 08:40:35 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ECD6F20673 for ; Tue, 8 Oct 2019 08:40:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ECD6F20673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46nW5D0DDgzDqFM for ; Tue, 8 Oct 2019 19:40:32 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=huawei.com (client-ip=45.249.212.35; helo=huawei.com; envelope-from=linyunsheng@huawei.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=huawei.com Received: from huawei.com (szxga07-in.huawei.com [45.249.212.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46nW3C2gW9zDqDY for ; Tue, 8 Oct 2019 19:38:45 +1100 (AEDT) Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 52ADD745F3E9E7BF8522; Tue, 8 Oct 2019 16:38:39 +0800 (CST) Received: from [127.0.0.1] (10.74.191.121) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.439.0; Tue, 8 Oct 2019 16:38:37 +0800 Subject: Re: [PATCH v6] numa: make node_to_cpumask_map() NUMA_NO_NODE aware To: Peter Zijlstra References: <20190924091714.GJ2369@hirez.programming.kicks-ass.net> <20190924105622.GH23050@dhcp22.suse.cz> <20190924112349.GJ2332@hirez.programming.kicks-ass.net> <20190924115401.GM23050@dhcp22.suse.cz> <20190924120943.GP2349@hirez.programming.kicks-ass.net> <20190924122500.GP23050@dhcp22.suse.cz> <20190924124325.GQ2349@hirez.programming.kicks-ass.net> <20190924125936.GR2349@hirez.programming.kicks-ass.net> <20190924131939.GS23050@dhcp22.suse.cz> <1adcbe68-6753-3497-48a0-cc84ac503372@huawei.com> <20190925104108.GE4553@hirez.programming.kicks-ass.net> From: Yunsheng Lin Message-ID: <47fa4cee-8528-7c23-c7de-7be1b65aa2ae@huawei.com> Date: Tue, 8 Oct 2019 16:38:28 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20190925104108.GE4553@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.74.191.121] X-CFilter-Loop: Reflected X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: dalias@libc.org, linux-sh@vger.kernel.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, heiko.carstens@de.ibm.com, jiaxun.yang@flygoat.com, Michal Hocko , mwb@linux.vnet.ibm.com, paulus@samba.org, hpa@zytor.com, sparclinux@vger.kernel.org, chenhc@lemote.com, will@kernel.org, cai@lca.pw, linux-s390@vger.kernel.org, ysato@users.sourceforge.jp, x86@kernel.org, rppt@linux.ibm.com, borntraeger@de.ibm.com, dledford@redhat.com, mingo@redhat.com, jeffrey.t.kirsher@intel.com, jhogan@kernel.org, mattst88@gmail.com, linux-mips@vger.kernel.org, len.brown@intel.com, gor@linux.ibm.com, anshuman.khandual@arm.com, gregkh@linuxfoundation.org, bp@alien8.de, luto@kernel.org, tglx@linutronix.de, naveen.n.rao@linux.vnet.ibm.com, linux-arm-kernel@lists.infradead.org, rth@twiddle.net, axboe@kernel.dk, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, ralf@linux-mips.org, tbogendoerfer@suse.de, paul.burton@mips.com, linux-alpha@vger.kernel.org, rafael@kernel.org, ink@jurassic.park.msu.ru, akpm@linux-foundation.org, robin.murphy@arm.com, davem@davemloft.net Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On 2019/9/25 18:41, Peter Zijlstra wrote: > On Wed, Sep 25, 2019 at 05:14:20PM +0800, Yunsheng Lin wrote: >> From the discussion above, It seems making the node_to_cpumask_map() >> NUMA_NO_NODE aware is the most feasible way to move forwad. > > That's still wrong. Hi, Peter It seems this has trapped in the dead circle. >From my understanding, NUMA_NO_NODE which means not node numa preference is the state to describe the node of virtual device or the physical device that has equal distance to all cpu. We can be stricter if the device does have a nearer node, but we can not deny that a device does not have a node numa preference or node affinity, which also means the control or data buffer can be allocated at the node where the process is running. As you has proposed, making it -2 and have dev_to_node() warn if the device does have a nearer node and not set by the fw is a way to be stricter. But I think maybe being stricter is not really relevant to NUMA_NO_NODE, because we does need a state to describe the device that have equal distance to all node, even if it is not physically scalable. Any better suggestion to move this forward? > > . > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yunsheng Lin Subject: Re: [PATCH v6] numa: make node_to_cpumask_map() NUMA_NO_NODE aware Date: Tue, 8 Oct 2019 16:38:28 +0800 Message-ID: <47fa4cee-8528-7c23-c7de-7be1b65aa2ae@huawei.com> References: <20190924091714.GJ2369@hirez.programming.kicks-ass.net> <20190924105622.GH23050@dhcp22.suse.cz> <20190924112349.GJ2332@hirez.programming.kicks-ass.net> <20190924115401.GM23050@dhcp22.suse.cz> <20190924120943.GP2349@hirez.programming.kicks-ass.net> <20190924122500.GP23050@dhcp22.suse.cz> <20190924124325.GQ2349@hirez.programming.kicks-ass.net> <20190924125936.GR2349@hirez.programming.kicks-ass.net> <20190924131939.GS23050@dhcp22.suse.cz> <1adcbe68-6753-3497-48a0-cc84ac503372@huawei.com> <20190925104108.GE4553@hirez.programming.kicks-ass.net> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20190925104108.GE4553@hirez.programming.kicks-ass.net> Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii" To: Peter Zijlstra Cc: Michal Hocko , catalin.marinas@arm.com, will@kernel.org, mingo@redhat.com, bp@alien8.de, rth@twiddle.net, ink@jurassic.park.msu.ru, mattst88@gmail.com, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, heiko.carstens@de.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, ysato@users.sourceforge.jp, dalias@libc.org, davem@davemloft.net, ralf@linux-mips.org, paul.burton@mips.com, jhogan@kernel.org, jiaxun.yang@flygoat.com, chenhc@lemote.com, akpm@linux-foundation.org, rppt@linux.ibm.com, anshuman.khandual@arm.com, tglx@linutronix.de, cai@lca.pw, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, hpa@zytor.com, x86@kernel.org, dave.hansen@linux On 2019/9/25 18:41, Peter Zijlstra wrote: > On Wed, Sep 25, 2019 at 05:14:20PM +0800, Yunsheng Lin wrote: >> From the discussion above, It seems making the node_to_cpumask_map() >> NUMA_NO_NODE aware is the most feasible way to move forwad. > > That's still wrong. Hi, Peter It seems this has trapped in the dead circle. >From my understanding, NUMA_NO_NODE which means not node numa preference is the state to describe the node of virtual device or the physical device that has equal distance to all cpu. We can be stricter if the device does have a nearer node, but we can not deny that a device does not have a node numa preference or node affinity, which also means the control or data buffer can be allocated at the node where the process is running. As you has proposed, making it -2 and have dev_to_node() warn if the device does have a nearer node and not set by the fw is a way to be stricter. But I think maybe being stricter is not really relevant to NUMA_NO_NODE, because we does need a state to describe the device that have equal distance to all node, even if it is not physically scalable. Any better suggestion to move this forward? > > . >