[PATCH] riscv: cpu-hotplug: clear cpu from numa map when teardown

[PATCH] riscv: cpu-hotplug: clear cpu from numa map when teardown

[Pingfan Liu]

There is numa_add_cpu() when cpus online, accordingly, there should be
numa_remove_cpu() when cpus offline.

Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Jisheng Zhang <jszhang@kernel.org>
To: linux-riscv@lists.infradead.org
---
 arch/riscv/kernel/cpu-hotplug.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
index be7f05b542bb..78d156a359ef 100644
--- a/arch/riscv/kernel/cpu-hotplug.c
+++ b/arch/riscv/kernel/cpu-hotplug.c
@@ -40,6 +40,7 @@ int __cpu_disable(void)
 		return ret;
 
 	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
 	set_cpu_online(cpu, false);
 	irq_migrate_all_off_this_cpu();
 
-- 
2.31.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH] riscv: cpu-hotplug: clear cpu from numa map when teardown

[Palmer Dabbelt]

On Sun, 23 Jan 2022 04:13:52 PST (-0800), kernelfans@gmail.com wrote:
> There is numa_add_cpu() when cpus online, accordingly, there should be
> numa_remove_cpu() when cpus offline.
>
> Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
> Cc: Paul Walmsley <paul.walmsley@sifive.com>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: Albert Ou <aou@eecs.berkeley.edu>
> Cc: Jisheng Zhang <jszhang@kernel.org>
> To: linux-riscv@lists.infradead.org
> ---
>  arch/riscv/kernel/cpu-hotplug.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
> index be7f05b542bb..78d156a359ef 100644
> --- a/arch/riscv/kernel/cpu-hotplug.c
> +++ b/arch/riscv/kernel/cpu-hotplug.c
> @@ -40,6 +40,7 @@ int __cpu_disable(void)
>  		return ret;
>
>  	remove_cpu_topology(cpu);
> +	numa_remove_cpu(cpu);
>  	set_cpu_online(cpu, false);
>  	irq_migrate_all_off_this_cpu();

Thanks, this is on fixes.  I don't really have any NUMA-related test 
setups, but if someone has a QEMU-based config that's meaningful then 
I'm happy to add it.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH] riscv: cpu-hotplug: clear cpu from numa map when teardown

[Pingfan Liu]

On Wed, Feb 9, 2022 at 9:13 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Sun, 23 Jan 2022 04:13:52 PST (-0800), kernelfans@gmail.com wrote:
> > There is numa_add_cpu() when cpus online, accordingly, there should be
> > numa_remove_cpu() when cpus offline.
> >
> > Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
> > Cc: Paul Walmsley <paul.walmsley@sifive.com>
> > Cc: Palmer Dabbelt <palmer@dabbelt.com>
> > Cc: Albert Ou <aou@eecs.berkeley.edu>
> > Cc: Jisheng Zhang <jszhang@kernel.org>
> > To: linux-riscv@lists.infradead.org
> > ---
> >  arch/riscv/kernel/cpu-hotplug.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
> > index be7f05b542bb..78d156a359ef 100644
> > --- a/arch/riscv/kernel/cpu-hotplug.c
> > +++ b/arch/riscv/kernel/cpu-hotplug.c
> > @@ -40,6 +40,7 @@ int __cpu_disable(void)
> >               return ret;
> >
> >       remove_cpu_topology(cpu);
> > +     numa_remove_cpu(cpu);
> >       set_cpu_online(cpu, false);
> >       irq_migrate_all_off_this_cpu();
>
> Thanks, this is on fixes.  I don't really have any NUMA-related test
> setups, but if someone has a QEMU-based config that's meaningful then
> I'm happy to add it.

Sorry that I have no such things even riscv machine. I did some
hotplug stuff, compared arm/arm64/riscv, and found the difference.

But since the code path should be symmetric, I think the hot-removing
path should revert the hot-adding path.

Thanks,

Pingfan

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH] riscv: cpu-hotplug: clear cpu from numa map when teardown

[Palmer Dabbelt]

On Tue, 08 Feb 2022 17:13:21 PST (-0800), Palmer Dabbelt wrote:
> On Sun, 23 Jan 2022 04:13:52 PST (-0800), kernelfans@gmail.com wrote:
>> There is numa_add_cpu() when cpus online, accordingly, there should be
>> numa_remove_cpu() when cpus offline.
>>
>> Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
>> Cc: Paul Walmsley <paul.walmsley@sifive.com>
>> Cc: Palmer Dabbelt <palmer@dabbelt.com>
>> Cc: Albert Ou <aou@eecs.berkeley.edu>
>> Cc: Jisheng Zhang <jszhang@kernel.org>
>> To: linux-riscv@lists.infradead.org
>> ---
>>  arch/riscv/kernel/cpu-hotplug.c | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
>> index be7f05b542bb..78d156a359ef 100644
>> --- a/arch/riscv/kernel/cpu-hotplug.c
>> +++ b/arch/riscv/kernel/cpu-hotplug.c
>> @@ -40,6 +40,7 @@ int __cpu_disable(void)
>>  		return ret;
>>
>>  	remove_cpu_topology(cpu);
>> +	numa_remove_cpu(cpu);
>>  	set_cpu_online(cpu, false);
>>  	irq_migrate_all_off_this_cpu();
>
> Thanks, this is on fixes.  I don't really have any NUMA-related test
> setups, but if someone has a QEMU-based config that's meaningful then
> I'm happy to add it.

I think I accidantally pushed this to the public branch rather than my 
staging branch, so I missed a build issue (missing numa include in some 
configurations).  I've force pushed a version of the patch with the fix, 
it's included here for reference -- I generally don't like to force push 
fixes, but given that it's a pretty trivial build failure I think that's 
the best way to go here.

commit f40fe31c01445f31253b15bef2412b33ae31093b
gpg: Signature made Thu 10 Feb 2022 09:17:00 AM PST
gpg:                using RSA key 2B3C3747446843B24A943A7A2E1319F35FBB1889
gpg:                issuer "palmer@dabbelt.com"
gpg: Good signature from "Palmer Dabbelt <palmer@dabbelt.com>" [ultimate]
gpg:                 aka "Palmer Dabbelt <palmer@rivosinc.com>" [ultimate]
Author: Pingfan Liu <kernelfans@gmail.com>
Date:   Sun Jan 23 20:13:52 2022 +0800

    riscv: cpu-hotplug: clear cpu from numa map when teardown

    There is numa_add_cpu() when cpus online, accordingly, there should be
    numa_remove_cpu() when cpus offline.

    Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
    Fixes: 4f0e8eef772e ("riscv: Add numa support for riscv64 platform")
    Cc: stable@vger.kernel.org
    [Palmer: Add missing NUMA include]
    Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
index be7f05b542bb..f7a832e3a1d1 100644
--- a/arch/riscv/kernel/cpu-hotplug.c
+++ b/arch/riscv/kernel/cpu-hotplug.c
@@ -12,6 +12,7 @@
 #include <linux/sched/hotplug.h>
 #include <asm/irq.h>
 #include <asm/cpu_ops.h>
+#include <asm/numa.h>
 #include <asm/sbi.h>

 bool cpu_has_hotplug(unsigned int cpu)
@@ -40,6 +41,7 @@ int __cpu_disable(void)
 		return ret;

 	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
 	set_cpu_online(cpu, false);
 	irq_migrate_all_off_this_cpu();



_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH] riscv: cpu-hotplug: clear cpu from numa map when teardown

[Atish Patra]

On Tue, Feb 8, 2022 at 5:13 PM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Sun, 23 Jan 2022 04:13:52 PST (-0800), kernelfans@gmail.com wrote:
> > There is numa_add_cpu() when cpus online, accordingly, there should be
> > numa_remove_cpu() when cpus offline.
> >
> > Signed-off-by: Pingfan Liu <kernelfans@gmail.com>
> > Cc: Paul Walmsley <paul.walmsley@sifive.com>
> > Cc: Palmer Dabbelt <palmer@dabbelt.com>
> > Cc: Albert Ou <aou@eecs.berkeley.edu>
> > Cc: Jisheng Zhang <jszhang@kernel.org>
> > To: linux-riscv@lists.infradead.org
> > ---
> >  arch/riscv/kernel/cpu-hotplug.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/riscv/kernel/cpu-hotplug.c b/arch/riscv/kernel/cpu-hotplug.c
> > index be7f05b542bb..78d156a359ef 100644
> > --- a/arch/riscv/kernel/cpu-hotplug.c
> > +++ b/arch/riscv/kernel/cpu-hotplug.c
> > @@ -40,6 +40,7 @@ int __cpu_disable(void)
> >               return ret;
> >
> >       remove_cpu_topology(cpu);
> > +     numa_remove_cpu(cpu);
> >       set_cpu_online(cpu, false);
> >       irq_migrate_all_off_this_cpu();
>
> Thanks, this is on fixes.  I don't really have any NUMA-related test
> setups, but if someone has a QEMU-based config that's meaningful then
> I'm happy to add it.
>

lscpu/numactl already checks the online cpu list and prints correct output
without the fix which explains why I did not notice this earlier.

This is the correct thing to do anyways. Thanks for the fix.

Tested-by: Atish Patra <atishp@rivosinc.com>

Here is my qemu config
qemu-system-riscv64 -cpu rv64 -M virt -m 1G -smp 4 \
-objectmemory-backend-ram,size=512M,policy=bind,host-nodes=0,id=ram-node0
\
-numa node,memdev=ram-node0 \
-object memory-backend-ram,size=512M,policy=bind,host-nodes=0,id=ram-node1 \
-numa node,memdev=ram-node1

> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv



-- 
Regards,
Atish

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv