linuxppc-dev.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core
@ 2023-12-29 12:01 Michael Ellerman
  2023-12-29 12:01 ` [RFC PATCH 2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU Michael Ellerman
                   ` (4 more replies)
  0 siblings, 5 replies; 15+ messages in thread
From: Michael Ellerman @ 2023-12-29 12:01 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: Hari Bathini, Pingfan Liu, Pingfan Liu

If nr_cpu_ids is too low to include at least all the threads of a single
core adjust nr_cpu_ids upwards. This avoids triggering odd bugs in code
that assumes all threads of a core are available.

Cc: stable@vger.kernel.org
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/kernel/prom.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 0b5878c3125b..58e80076bed5 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -375,6 +375,12 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
 	if (IS_ENABLED(CONFIG_PPC64))
 		boot_cpu_hwid = be32_to_cpu(intserv[found_thread]);
 
+	if (nr_cpu_ids % nthreads != 0) {
+		set_nr_cpu_ids(ALIGN(nr_cpu_ids, nthreads));
+		pr_warn("nr_cpu_ids was not a multiple of threads_per_core, adjusted to %d\n",
+			nr_cpu_ids);
+	}
+
 	/*
 	 * PAPR defines "logical" PVR values for cpus that
 	 * meet various levels of the architecture:
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC PATCH 2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU
  2023-12-29 12:01 [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
@ 2023-12-29 12:01 ` Michael Ellerman
  2023-12-29 12:01 ` [RFC PATCH 3/5] powerpc/smp: Lookup avail once per device tree node Michael Ellerman
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 15+ messages in thread
From: Michael Ellerman @ 2023-12-29 12:01 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: Hari Bathini, Pingfan Liu, Pingfan Liu

If nr_cpu_ids is too low to include the boot CPU adjust nr_cpu_ids
upward. Otherwise the kernel will BUG when trying to allocate a paca
for the boot CPU and fail to boot.

Cc: stable@vger.kernel.org
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/kernel/prom.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 58e80076bed5..77364729a1b6 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -381,6 +381,12 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
 			nr_cpu_ids);
 	}
 
+	if (boot_cpuid >= nr_cpu_ids) {
+		set_nr_cpu_ids(min(CONFIG_NR_CPUS, ALIGN(boot_cpuid + 1, nthreads)));
+		pr_warn("Boot CPU %d >= nr_cpu_ids, adjusted nr_cpu_ids to %d\n",
+			boot_cpuid, nr_cpu_ids);
+	}
+
 	/*
 	 * PAPR defines "logical" PVR values for cpus that
 	 * meet various levels of the architecture:
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC PATCH 3/5] powerpc/smp: Lookup avail once per device tree node
  2023-12-29 12:01 [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
  2023-12-29 12:01 ` [RFC PATCH 2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU Michael Ellerman
@ 2023-12-29 12:01 ` Michael Ellerman
  2023-12-29 12:01 ` [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads() Michael Ellerman
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 15+ messages in thread
From: Michael Ellerman @ 2023-12-29 12:01 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: Hari Bathini, Pingfan Liu, Pingfan Liu

The of_device_is_available() check only needs to be done once per device
node, there's no need to repeat it for each thread. Move it out of the
loop.

Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/kernel/setup-common.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index ec02f9d8f55d..ff7616023ade 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -468,17 +468,16 @@ void __init smp_setup_cpu_maps(void)
 
 		nthreads = len / sizeof(int);
 
+		bool avail = of_device_is_available(dn);
+		if (!avail)
+			avail = !of_property_match_string(dn,
+					"enable-method", "spin-table");
+
 		for (j = 0; j < nthreads && cpu < nr_cpu_ids; j++) {
-			bool avail;
 
 			DBG("    thread %d -> cpu %d (hard id %d)\n",
 			    j, cpu, be32_to_cpu(intserv[j]));
 
-			avail = of_device_is_available(dn);
-			if (!avail)
-				avail = !of_property_match_string(dn,
-						"enable-method", "spin-table");
-
 			set_cpu_present(cpu, avail);
 			set_cpu_possible(cpu, true);
 			cpu_to_phys_id[cpu] = be32_to_cpu(intserv[j]);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads()
  2023-12-29 12:01 [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
  2023-12-29 12:01 ` [RFC PATCH 2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU Michael Ellerman
  2023-12-29 12:01 ` [RFC PATCH 3/5] powerpc/smp: Lookup avail once per device tree node Michael Ellerman
@ 2023-12-29 12:01 ` Michael Ellerman
  2024-01-02  4:34   ` Aneesh Kumar K.V
  2023-12-29 12:01 ` [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids Michael Ellerman
  2024-02-15 13:00 ` [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
  4 siblings, 1 reply; 15+ messages in thread
From: Michael Ellerman @ 2023-12-29 12:01 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: Hari Bathini, Pingfan Liu, Pingfan Liu

Factor out the for loop that assigns CPU numbers to threads of a core.
The function takes the next CPU number to use as input, and returns the
next available CPU number after the threads has been assigned.

This will allow a subsequent change to assign threads out of order.

Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/kernel/setup-common.c | 32 ++++++++++++++++++++----------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index ff7616023ade..d9f8ed9bd2fc 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -411,6 +411,25 @@ static void __init cpu_init_thread_core_maps(int tpc)
 
 u32 *cpu_to_phys_id = NULL;
 
+static int assign_threads(unsigned cpu, unsigned int nthreads, bool avail,
+                          const __be32 *hw_ids)
+{
+	for (int i = 0; i < nthreads && cpu < nr_cpu_ids; i++) {
+		__be32 hwid;
+
+		hwid = be32_to_cpu(hw_ids[i]);
+
+		DBG("    thread %d -> cpu %d (hard id %d)\n", i, cpu, hwid);
+
+		set_cpu_present(cpu, avail);
+		set_cpu_possible(cpu, true);
+		cpu_to_phys_id[cpu] = hwid;
+		cpu++;
+	}
+
+	return cpu;
+}
+
 /**
  * setup_cpu_maps - initialize the following cpu maps:
  *                  cpu_possible_mask
@@ -446,7 +465,7 @@ void __init smp_setup_cpu_maps(void)
 	for_each_node_by_type(dn, "cpu") {
 		const __be32 *intserv;
 		__be32 cpu_be;
-		int j, len;
+		int len;
 
 		DBG("  * %pOF...\n", dn);
 
@@ -473,16 +492,7 @@ void __init smp_setup_cpu_maps(void)
 			avail = !of_property_match_string(dn,
 					"enable-method", "spin-table");
 
-		for (j = 0; j < nthreads && cpu < nr_cpu_ids; j++) {
-
-			DBG("    thread %d -> cpu %d (hard id %d)\n",
-			    j, cpu, be32_to_cpu(intserv[j]));
-
-			set_cpu_present(cpu, avail);
-			set_cpu_possible(cpu, true);
-			cpu_to_phys_id[cpu] = be32_to_cpu(intserv[j]);
-			cpu++;
-		}
+		cpu = assign_threads(cpu, nthreads, avail, intserv);
 
 		if (cpu >= nr_cpu_ids) {
 			of_node_put(dn);
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2023-12-29 12:01 [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
                   ` (2 preceding siblings ...)
  2023-12-29 12:01 ` [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads() Michael Ellerman
@ 2023-12-29 12:01 ` Michael Ellerman
  2023-12-29 12:07   ` Michael Ellerman
  2024-01-02  4:46   ` Aneesh Kumar K.V
  2024-02-15 13:00 ` [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
  4 siblings, 2 replies; 15+ messages in thread
From: Michael Ellerman @ 2023-12-29 12:01 UTC (permalink / raw)
  To: linuxppc-dev; +Cc: Hari Bathini, Pingfan Liu, Pingfan Liu

If nr_cpu_ids is too low to include the boot CPU, remap the boot CPU
onto logical core 0.

This is achieved in two stages. In early_init_dt_scan_cpus() the boot
CPU is renumbered to be on logical core 0, and the original boot core's
hardware ID is recorded.

Later in smp_setup_cpu_maps(), if the original boot core ID is set, the
logical CPU numbers on the 0th core are skipped in the normal device
tree search over CPU device tree nodes. Then the search is continued
until the device tree node matching the boot core is found, and those
CPUs are assigned the CPU numbers starting at 0.

This allows kdump kernels to be booted with low values for nr_cpu_ids
to conserve memory, while also allowing the crashing/boot CPU to be
any CPU.

Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/include/asm/smp.h     |  1 +
 arch/powerpc/kernel/prom.c         | 16 +++++++++++-----
 arch/powerpc/kernel/setup-common.c | 19 +++++++++++++++++--
 3 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/arch/powerpc/include/asm/smp.h b/arch/powerpc/include/asm/smp.h
index aaaa576d0e15..b77927ccb0ab 100644
--- a/arch/powerpc/include/asm/smp.h
+++ b/arch/powerpc/include/asm/smp.h
@@ -27,6 +27,7 @@
 
 extern int boot_cpuid;
 extern int boot_cpu_hwid; /* PPC64 only */
+extern int boot_core_hwid;
 extern int spinning_secondaries;
 extern u32 *cpu_to_phys_id;
 extern bool coregroup_enabled;
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 77364729a1b6..af2b70585b2e 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -368,8 +368,6 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
 	if (found < 0)
 		return 0;
 
-	DBG("boot cpu: logical %d physical %d\n", found,
-	    be32_to_cpu(intserv[found_thread]));
 	boot_cpuid = found;
 
 	if (IS_ENABLED(CONFIG_PPC64))
@@ -382,11 +380,19 @@ static int __init early_init_dt_scan_cpus(unsigned long node,
 	}
 
 	if (boot_cpuid >= nr_cpu_ids) {
-		set_nr_cpu_ids(min(CONFIG_NR_CPUS, ALIGN(boot_cpuid + 1, nthreads)));
-		pr_warn("Boot CPU %d >= nr_cpu_ids, adjusted nr_cpu_ids to %d\n",
-			boot_cpuid, nr_cpu_ids);
+		// Remember boot core for smp_setup_cpu_maps()
+		boot_core_hwid = be32_to_cpu(intserv[0]);
+		
+		pr_warn("Boot CPU %d (core hwid %d) >= nr_cpu_ids, adjusted boot CPU to %d\n",
+			boot_cpuid, boot_core_hwid, found_thread);
+
+		// Adjust boot CPU to appear on logical core 0
+		boot_cpuid = found_thread;
 	}
 
+	DBG("boot cpu: logical %d physical %d\n", boot_cpuid,
+	    be32_to_cpu(intserv[found_thread]));
+
 	/*
 	 * PAPR defines "logical" PVR values for cpus that
 	 * meet various levels of the architecture:
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index d9f8ed9bd2fc..5844f3d113a5 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -85,6 +85,7 @@ EXPORT_SYMBOL(machine_id);
 
 int boot_cpuid = -1;
 EXPORT_SYMBOL_GPL(boot_cpuid);
+int __initdata boot_core_hwid = -1;
 
 #ifdef CONFIG_PPC64
 int boot_cpu_hwid = -1;
@@ -492,12 +493,26 @@ void __init smp_setup_cpu_maps(void)
 			avail = !of_property_match_string(dn,
 					"enable-method", "spin-table");
 
-		cpu = assign_threads(cpu, nthreads, avail, intserv);
+		if (boot_core_hwid >= 0) {
+			if (cpu == 0) {
+				pr_info("Skipping CPU node %pOF to allow for boot core.\n", dn);
+				cpu = nthreads;
+				continue;
+			}
 
-		if (cpu >= nr_cpu_ids) {
+			if (be32_to_cpu(intserv[0]) == boot_core_hwid) {
+				pr_info("Renumbered boot core %pOF to logical 0\n", dn);
+				assign_threads(0, nthreads, avail, intserv);
+				of_node_put(dn);
+				break;
+			}
+		} else if (cpu >= nr_cpu_ids) {
 			of_node_put(dn);
 			break;
 		}
+
+		if (cpu < nr_cpu_ids)
+			cpu = assign_threads(cpu, nthreads, avail, intserv);
 	}
 
 	/* If no SMT supported, nthreads is forced to 1 */
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2023-12-29 12:01 ` [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids Michael Ellerman
@ 2023-12-29 12:07   ` Michael Ellerman
  2024-01-02  0:51     ` Pingfan Liu
  2024-01-02  4:46   ` Aneesh Kumar K.V
  1 sibling, 1 reply; 15+ messages in thread
From: Michael Ellerman @ 2023-12-29 12:07 UTC (permalink / raw)
  To: Pingfan Liu, Pingfan Liu, Hari Bathini; +Cc: linuxppc-dev

Michael Ellerman <mpe@ellerman.id.au> writes:
> If nr_cpu_ids is too low to include the boot CPU, remap the boot CPU
> onto logical core 0.

Hi guys,

I finally got time to look at this issue. I think this series should fix
the problems that have been seen. I've tested this fairly thoroughly
with a qemu script, and also a few boots on a real machine.

If you can test it with your setups that would be great. Hopefully there
isn't some obscure case I've missed.

cheers

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2023-12-29 12:07   ` Michael Ellerman
@ 2024-01-02  0:51     ` Pingfan Liu
  2024-02-13 20:16       ` Wen Xiong
  0 siblings, 1 reply; 15+ messages in thread
From: Pingfan Liu @ 2024-01-02  0:51 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Ming Lei, linuxppc-dev, Hari Bathini, Pingfan Liu, Wen Xiong

On Fri, Dec 29, 2023 at 8:07 PM Michael Ellerman <mpe@ellerman.id.au> wrote:
>
> Michael Ellerman <mpe@ellerman.id.au> writes:
> > If nr_cpu_ids is too low to include the boot CPU, remap the boot CPU
> > onto logical core 0.
>
> Hi guys,
>
> I finally got time to look at this issue. I think this series should fix

Thanks a lot for sparing time on it and hope we can close this
prolonged issue soon.

And loop in Wen Xiong and Ming Lei, who care for this issue too.

Best Regards,

Pingfan

> the problems that have been seen. I've tested this fairly thoroughly
> with a qemu script, and also a few boots on a real machine.
>
> If you can test it with your setups that would be great. Hopefully there
> isn't some obscure case I've missed.
>
> cheers
>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads()
  2023-12-29 12:01 ` [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads() Michael Ellerman
@ 2024-01-02  4:34   ` Aneesh Kumar K.V
  2024-02-14 13:10     ` Michael Ellerman
  0 siblings, 1 reply; 15+ messages in thread
From: Aneesh Kumar K.V @ 2024-01-02  4:34 UTC (permalink / raw)
  To: Michael Ellerman, linuxppc-dev; +Cc: Pingfan Liu, Pingfan Liu, Hari Bathini

Michael Ellerman <mpe@ellerman.id.au> writes:

....
  
> +static int assign_threads(unsigned cpu, unsigned int nthreads, bool avail,
>

May be rename 'avail' to 'present'

> +                          const __be32 *hw_ids)
> +{
> +	for (int i = 0; i < nthreads && cpu < nr_cpu_ids; i++) {
> +		__be32 hwid;
> +
> +		hwid = be32_to_cpu(hw_ids[i]);
> +
> +		DBG("    thread %d -> cpu %d (hard id %d)\n", i, cpu, hwid);
> +
> +		set_cpu_present(cpu, avail);
> +		set_cpu_possible(cpu, true);
> +		cpu_to_phys_id[cpu] = hwid;
> +		cpu++;
> +	}
> +

-aneesh

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2023-12-29 12:01 ` [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids Michael Ellerman
  2023-12-29 12:07   ` Michael Ellerman
@ 2024-01-02  4:46   ` Aneesh Kumar K.V
  2024-02-09 15:56     ` Jiri Bohac
  1 sibling, 1 reply; 15+ messages in thread
From: Aneesh Kumar K.V @ 2024-01-02  4:46 UTC (permalink / raw)
  To: Michael Ellerman, linuxppc-dev; +Cc: Pingfan Liu, Pingfan Liu, Hari Bathini

Michael Ellerman <mpe@ellerman.id.au> writes:

....

>  #ifdef CONFIG_PPC64
>  int boot_cpu_hwid = -1;
> @@ -492,12 +493,26 @@ void __init smp_setup_cpu_maps(void)
>  			avail = !of_property_match_string(dn,
>  					"enable-method", "spin-table");
>  
> -		cpu = assign_threads(cpu, nthreads, avail, intserv);
> +		if (boot_core_hwid >= 0) {
> +			if (cpu == 0) {
> +				pr_info("Skipping CPU node %pOF to allow for boot core.\n", dn);
> +				cpu = nthreads;
> +				continue;
> +			}
>  
> -		if (cpu >= nr_cpu_ids) {
> +			if (be32_to_cpu(intserv[0]) == boot_core_hwid) {
> +				pr_info("Renumbered boot core %pOF to logical 0\n", dn);
> +				assign_threads(0, nthreads, avail, intserv);
> +				of_node_put(dn);
> +				break;
>

I was expecting a 'continue' here. Why 'break' the loop? The condition that
should break the loop should be cpu >= nr_cpu_ids 


> +			}
> +		} else if (cpu >= nr_cpu_ids) {
>  			of_node_put(dn);
>  			break;
>  		}
> +
> +		if (cpu < nr_cpu_ids)
> +			cpu = assign_threads(cpu, nthreads, avail, intserv);
>  	}
>  
>  	/* If no SMT supported, nthreads is forced to 1 */
> -- 
> 2.43.0

-aneesh

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2024-01-02  4:46   ` Aneesh Kumar K.V
@ 2024-02-09 15:56     ` Jiri Bohac
  2024-02-14 13:12       ` Michael Ellerman
  0 siblings, 1 reply; 15+ messages in thread
From: Jiri Bohac @ 2024-02-09 15:56 UTC (permalink / raw)
  To: Aneesh Kumar K.V; +Cc: Hari Bathini, linuxppc-dev, Pingfan Liu, Pingfan Liu

On Tue, Jan 02, 2024 at 10:16:04AM +0530, Aneesh Kumar K.V wrote:
> Michael Ellerman <mpe@ellerman.id.au> writes:
> 
> ....
> 
> >  #ifdef CONFIG_PPC64
> >  int boot_cpu_hwid = -1;
> > @@ -492,12 +493,26 @@ void __init smp_setup_cpu_maps(void)
> >  			avail = !of_property_match_string(dn,
> >  					"enable-method", "spin-table");
> >  
> > -		cpu = assign_threads(cpu, nthreads, avail, intserv);
> > +		if (boot_core_hwid >= 0) {
> > +			if (cpu == 0) {
> > +				pr_info("Skipping CPU node %pOF to allow for boot core.\n", dn);
> > +				cpu = nthreads;
> > +				continue;
> > +			}
> >  
> > -		if (cpu >= nr_cpu_ids) {
> > +			if (be32_to_cpu(intserv[0]) == boot_core_hwid) {
> > +				pr_info("Renumbered boot core %pOF to logical 0\n", dn);
> > +				assign_threads(0, nthreads, avail, intserv);
> > +				of_node_put(dn);
> > +				break;
> >
> 
> I was expecting a 'continue' here. Why 'break' the loop? The condition that
> should break the loop should be cpu >= nr_cpu_ids 

No, the patch seems correct:

We're in the "if (boot_core_hwid >= 0)" branch, meaning that it
was determined by early_init_dt_scan_cpus() that boot_cpuid >=
nr_cpu_ids. So we loop until we get to the boot CPU, so it can be
renumbered to 0. Once we do that we break, because we
know we are already past nr_cpu_ids - otherwise boot_core_hwid
would not be >= 0. 


> > +			}
> > +		} else if (cpu >= nr_cpu_ids) {
> >  			of_node_put(dn);
> >  			break;
> >  		}

Here is what you expected - in case the boot CPU was < nr_cpu_ids
we break as soon as nr_cpu_ids is reached.

> > +
> > +		if (cpu < nr_cpu_ids)

this ensures that CPUs between nr_cpu_ids and the boot CPU are
correctly ignored in case we're already past nr_cpu_ids and only
scanning further to find the boot CPU to be renumbered to 0

-- 
Jiri Bohac <jbohac@suse.cz>
SUSE Labs, Prague, Czechia


^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2024-01-02  0:51     ` Pingfan Liu
@ 2024-02-13 20:16       ` Wen Xiong
  0 siblings, 0 replies; 15+ messages in thread
From: Wen Xiong @ 2024-02-13 20:16 UTC (permalink / raw)
  To: Pingfan Liu, Michael Ellerman
  Cc: linuxppc-dev, Hari Bathini, Pingfan Liu, Ming Lei

>>And loop in Wen Xiong and Ming Lei, who care for this issue too.


Hi Pingfan, Thanks for your email!

Hi Michael,

Thanks for your new patchset!

In past month, Our several test team have modified /etc/sysconfig/kdump file with nr_cpus=1, triggered kdump over the following IO devices and  kdump works fine with generating vmcore file.

Test kernel:  the latest upstream kernel + your patchset
                      the latest rhel94 kernel + backport of your patchset

Test hardware: Nvme drives, FC adapter, NVmf-over-FC

Test device drivers: nvme, lpfc, nvme-fc

Thanks for your work! Please consideration for inclusion in upstream kernel.

Thanks for your help!
Wendy

> the problems that have been seen. I've tested this fairly thoroughly 
> with a qemu script, and also a few boots on a real machine.
>
> If you can test it with your setups that would be great. Hopefully 
> there isn't some obscure case I've missed.
>
> cheers
>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads()
  2024-01-02  4:34   ` Aneesh Kumar K.V
@ 2024-02-14 13:10     ` Michael Ellerman
  0 siblings, 0 replies; 15+ messages in thread
From: Michael Ellerman @ 2024-02-14 13:10 UTC (permalink / raw)
  To: Aneesh Kumar K.V, linuxppc-dev; +Cc: Pingfan Liu, Pingfan Liu, Hari Bathini

Aneesh Kumar K.V <aneesh.kumar@kernel.org> writes:
> Michael Ellerman <mpe@ellerman.id.au> writes:
>
> ....
>   
>> +static int assign_threads(unsigned cpu, unsigned int nthreads, bool avail,
>>
>
> May be rename 'avail' to 'present'

Yeah, will do.

cheers

>> +                          const __be32 *hw_ids)
>> +{
>> +	for (int i = 0; i < nthreads && cpu < nr_cpu_ids; i++) {
>> +		__be32 hwid;
>> +
>> +		hwid = be32_to_cpu(hw_ids[i]);
>> +
>> +		DBG("    thread %d -> cpu %d (hard id %d)\n", i, cpu, hwid);
>> +
>> +		set_cpu_present(cpu, avail);
>> +		set_cpu_possible(cpu, true);
>> +		cpu_to_phys_id[cpu] = hwid;
>> +		cpu++;
>> +	}
>> +
>
> -aneesh

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
  2024-02-09 15:56     ` Jiri Bohac
@ 2024-02-14 13:12       ` Michael Ellerman
  0 siblings, 0 replies; 15+ messages in thread
From: Michael Ellerman @ 2024-02-14 13:12 UTC (permalink / raw)
  To: Jiri Bohac, Aneesh Kumar K.V
  Cc: Hari Bathini, linuxppc-dev, Pingfan Liu, Pingfan Liu

Jiri Bohac <jbohac@suse.cz> writes:
> On Tue, Jan 02, 2024 at 10:16:04AM +0530, Aneesh Kumar K.V wrote:
>> Michael Ellerman <mpe@ellerman.id.au> writes:
>> 
>> ....
>> 
>> >  #ifdef CONFIG_PPC64
>> >  int boot_cpu_hwid = -1;
>> > @@ -492,12 +493,26 @@ void __init smp_setup_cpu_maps(void)
>> >  			avail = !of_property_match_string(dn,
>> >  					"enable-method", "spin-table");
>> >  
>> > -		cpu = assign_threads(cpu, nthreads, avail, intserv);
>> > +		if (boot_core_hwid >= 0) {
>> > +			if (cpu == 0) {
>> > +				pr_info("Skipping CPU node %pOF to allow for boot core.\n", dn);
>> > +				cpu = nthreads;
>> > +				continue;
>> > +			}
>> >  
>> > -		if (cpu >= nr_cpu_ids) {
>> > +			if (be32_to_cpu(intserv[0]) == boot_core_hwid) {
>> > +				pr_info("Renumbered boot core %pOF to logical 0\n", dn);
>> > +				assign_threads(0, nthreads, avail, intserv);
>> > +				of_node_put(dn);
>> > +				break;
>> >
>> 
>> I was expecting a 'continue' here. Why 'break' the loop? The condition that
>> should break the loop should be cpu >= nr_cpu_ids 
>
> No, the patch seems correct:
>
> We're in the "if (boot_core_hwid >= 0)" branch, meaning that it
> was determined by early_init_dt_scan_cpus() that boot_cpuid >=
> nr_cpu_ids. So we loop until we get to the boot CPU, so it can be
> renumbered to 0. Once we do that we break, because we
> know we are already past nr_cpu_ids - otherwise boot_core_hwid
> would not be >= 0. 

Yes that's exactly right.

Thanks for answering for me (was on leave and still catching up).

cheers

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core
  2023-12-29 12:01 [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
                   ` (3 preceding siblings ...)
  2023-12-29 12:01 ` [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids Michael Ellerman
@ 2024-02-15 13:00 ` Michael Ellerman
  2024-02-16  2:42   ` Pingfan Liu
  4 siblings, 1 reply; 15+ messages in thread
From: Michael Ellerman @ 2024-02-15 13:00 UTC (permalink / raw)
  To: linuxppc-dev, Michael Ellerman; +Cc: Pingfan Liu, Pingfan Liu, Hari Bathini

On Fri, 29 Dec 2023 23:01:03 +1100, Michael Ellerman wrote:
> If nr_cpu_ids is too low to include at least all the threads of a single
> core adjust nr_cpu_ids upwards. This avoids triggering odd bugs in code
> that assumes all threads of a core are available.
> 
> 

Applied to powerpc/next.

[1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core
      https://git.kernel.org/powerpc/c/5580e96dad5a439d561d9648ffcbccb739c2a120
[2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU
      https://git.kernel.org/powerpc/c/777f81f0a9c780a6443bcf2c7785f0cc2e87c1ef
[3/5] powerpc/smp: Lookup avail once per device tree node
      https://git.kernel.org/powerpc/c/dca79603fbc592ec7ea8bd7ba274052d3984e882
[4/5] powerpc/smp: Factor out assign_threads()
      https://git.kernel.org/powerpc/c/9832de654499f0bf797a3719c4d4c5bd401f18f5
[5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
      https://git.kernel.org/powerpc/c/0875f1ceba974042069f04946aa8f1d4d1e688da

cheers

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core
  2024-02-15 13:00 ` [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
@ 2024-02-16  2:42   ` Pingfan Liu
  0 siblings, 0 replies; 15+ messages in thread
From: Pingfan Liu @ 2024-02-16  2:42 UTC (permalink / raw)
  To: Michael Ellerman; +Cc: linuxppc-dev, Pingfan Liu, Hari Bathini

On Thu, Feb 15, 2024 at 9:09 PM Michael Ellerman
<patch-notifications@ellerman.id.au> wrote:
>
> On Fri, 29 Dec 2023 23:01:03 +1100, Michael Ellerman wrote:
> > If nr_cpu_ids is too low to include at least all the threads of a single
> > core adjust nr_cpu_ids upwards. This avoids triggering odd bugs in code
> > that assumes all threads of a core are available.
> >
> >
>
> Applied to powerpc/next.
>

Great! After all these years, finally we are close to the conclusion
of this feature.

Thanks,

Pingfan

> [1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core
>       https://git.kernel.org/powerpc/c/5580e96dad5a439d561d9648ffcbccb739c2a120
> [2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU
>       https://git.kernel.org/powerpc/c/777f81f0a9c780a6443bcf2c7785f0cc2e87c1ef
> [3/5] powerpc/smp: Lookup avail once per device tree node
>       https://git.kernel.org/powerpc/c/dca79603fbc592ec7ea8bd7ba274052d3984e882
> [4/5] powerpc/smp: Factor out assign_threads()
>       https://git.kernel.org/powerpc/c/9832de654499f0bf797a3719c4d4c5bd401f18f5
> [5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids
>       https://git.kernel.org/powerpc/c/0875f1ceba974042069f04946aa8f1d4d1e688da
>
> cheers
>


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2024-02-16  2:43 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-29 12:01 [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
2023-12-29 12:01 ` [RFC PATCH 2/5] powerpc/smp: Increase nr_cpu_ids to include the boot CPU Michael Ellerman
2023-12-29 12:01 ` [RFC PATCH 3/5] powerpc/smp: Lookup avail once per device tree node Michael Ellerman
2023-12-29 12:01 ` [RFC PATCH 4/5] powerpc/smp: Factor out assign_threads() Michael Ellerman
2024-01-02  4:34   ` Aneesh Kumar K.V
2024-02-14 13:10     ` Michael Ellerman
2023-12-29 12:01 ` [RFC PATCH 5/5] powerpc/smp: Remap boot CPU onto core 0 if >= nr_cpu_ids Michael Ellerman
2023-12-29 12:07   ` Michael Ellerman
2024-01-02  0:51     ` Pingfan Liu
2024-02-13 20:16       ` Wen Xiong
2024-01-02  4:46   ` Aneesh Kumar K.V
2024-02-09 15:56     ` Jiri Bohac
2024-02-14 13:12       ` Michael Ellerman
2024-02-15 13:00 ` [RFC PATCH 1/5] powerpc/smp: Adjust nr_cpu_ids to cover all threads of a core Michael Ellerman
2024-02-16  2:42   ` Pingfan Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).