From: Robin Murphy <robin.murphy@arm.com>
To: Jonathan Cameron <Jonathan.Cameron@Huawei.com>,
Anshuman Khandual <anshuman.khandual@arm.com>
Cc: patches@amperecomputing.com,
Vanshidhar Konda <vanshikonda@os.amperecomputing.com>,
linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
Valentin Schneider <valentin.schneider@arm.com>
Subject: Re: [PATCH] arm64: NUMA: Kconfig: Increase max number of nodes
Date: Thu, 22 Oct 2020 00:44:15 +0100 [thread overview]
Message-ID: <da1b140c-fdd9-5970-fe83-02f33c3d8251@arm.com> (raw)
In-Reply-To: <20201021110201.00002092@Huawei.com>
On 2020-10-21 12:02, Jonathan Cameron wrote:
> On Wed, 21 Oct 2020 09:43:21 +0530
> Anshuman Khandual <anshuman.khandual@arm.com> wrote:
>
>> On 10/20/2020 11:39 PM, Valentin Schneider wrote:
>>>
>>> Hi,
>>>
>>> Nit on the subject: this only increases the default, the max is still 2¹⁰.
>>
>> Agreed.
>>
>>>
>>> On 20/10/20 18:34, Vanshidhar Konda wrote:
>>>> The current arm64 max NUMA nodes default to 4. Today's arm64 systems can
>>>> reach or exceed 16. Increase the number to 64 (matching x86_64).
>>>>
>>>> Signed-off-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com>
>>>> ---
>>>> arch/arm64/Kconfig | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>>> index 893130ce1626..3e69d3c981be 100644
>>>> --- a/arch/arm64/Kconfig
>>>> +++ b/arch/arm64/Kconfig
>>>> @@ -980,7 +980,7 @@ config NUMA
>>>> config NODES_SHIFT
>>>> int "Maximum NUMA Nodes (as a power of 2)"
>>>> range 1 10
>>>> - default "2"
>>>> + default "6"
>>>
>>> This leads to more statically allocated memory for things like node to CPU
>>> maps (see uses of MAX_NUMNODES), but that shouldn't be too much of an
>>> issue.
>>
>> The smaller systems should not be required to waste those memory in
>> a default case, unless there is a real and available larger system
>> with those increased nodes.
>>
>>>
>>> AIUI this also directly correlates to how many more page->flags bits are
>>> required: are we sure the max 10 works on any aarch64 platform? I'm
>>
>> We will have to test that. Besides 256 (2 ^ 8) is the first threshold
>> to be crossed here.
>>
>>> genuinely asking here, given that I'm mostly a stranger to the mm
>>> world. The default should be something we're somewhat confident works
>>> everywhere.
>>
>> Agreed. Do we really need to match X86 right now ? Do we really have
>> systems that has 64 nodes ? We should not increase the default node
>> value and then try to solve some new problems, when there might not
>> be any system which could even use that. I would suggest increase
>> NODES_SHIFT value upto as required by a real and available system.
>
> I'm not going to give precise numbers on near future systems but it is public
> that we ship 8 NUMA node ARM64 systems today. Things will get more
> interesting as CXL and CCIX enter the market on ARM systems,
> given chances are every CXL device will look like another NUMA
> node (CXL spec says they should be presented as such) and you
> may be able to rack up lots of them.
>
> So I'd argue minimum that makes sense today is 16 nodes, but looking forward
> even a little and 64 is not a great stretch.
> I'd make the jump to 64 so we can forget about this again for a year or two.
> People will want to run today's distros on these new machines and we'd
> rather not have to go around all the distros asking them to carry a patch
> increasing this count (I assume they are already carrying such a patch
> due to those 8 node systems)
Nit: I doubt any sane distro is going to carry a patch to adjust the
*default* value of a Kconfig option. They might tune the actual value in
their config, but, well, isn't that the whole point of configs? ;)
Robin.
>
> Jonathan
>
>>
>>>
>>>> depends on NEED_MULTIPLE_NODES
>>>> help
>>>> Specify the maximum number of NUMA Nodes available on the target
>>>
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-10-21 23:45 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-20 17:34 [PATCH] arm64: NUMA: Kconfig: Increase max number of nodes Vanshidhar Konda
2020-10-20 18:09 ` Valentin Schneider
2020-10-21 4:13 ` Anshuman Khandual
2020-10-21 11:02 ` Jonathan Cameron
2020-10-21 22:29 ` Valentin Schneider
2020-10-29 13:37 ` Catalin Marinas
2020-10-29 19:48 ` Vanshidhar Konda
2020-10-30 10:21 ` Catalin Marinas
2020-10-21 23:44 ` Robin Murphy [this message]
2020-10-22 1:07 ` Vanshi Konda
2020-10-22 11:21 ` Robin Murphy
2020-10-22 16:25 ` Vanshi Konda
2020-10-27 22:46 ` Dave Kleikamp
2020-10-27 23:13 ` Vanshidhar Konda
2020-10-27 23:14 ` Vanshidhar Konda
2020-10-21 16:02 ` Vanshi Konda
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=da1b140c-fdd9-5970-fe83-02f33c3d8251@arm.com \
--to=robin.murphy@arm.com \
--cc=Jonathan.Cameron@Huawei.com \
--cc=anshuman.khandual@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=patches@amperecomputing.com \
--cc=valentin.schneider@arm.com \
--cc=vanshikonda@os.amperecomputing.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).