* [PATCH] riscv: don't stop itself in smp_send_stop
@ 2018-11-29 16:32 Andreas Schwab
2018-11-29 18:52 ` Atish Patra
0 siblings, 1 reply; 7+ messages in thread
From: Andreas Schwab @ 2018-11-29 16:32 UTC (permalink / raw)
To: linux-riscv
Only run ipi_stop on other cpus, not itself. Don't wait for ipi_stop to
return, since it never does. Also, mark each cpu offline.
Signed-off-by: Andreas Schwab <schwab@suse.de>
---
arch/riscv/kernel/smp.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index 57b1383e5e..5c6bb5fe5a 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -23,6 +23,7 @@
#include <linux/smp.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
+#include <linux/delay.h>
#include <asm/sbi.h>
#include <asm/tlbflush.h>
@@ -148,13 +149,27 @@ void arch_send_call_function_single_ipi(int cpu)
static void ipi_stop(void *unused)
{
+ set_cpu_online (smp_processor_id(), false);
while (1)
wait_for_interrupt();
}
void smp_send_stop(void)
{
- on_each_cpu(ipi_stop, NULL, 1);
+ unsigned long timeout;
+ struct cpumask mask;
+
+ cpumask_copy (&mask, cpu_online_mask);
+ cpumask_clear_cpu (smp_processor_id (), &mask);
+ on_each_cpu_mask(&mask, ipi_stop, NULL, 0);
+
+ /* Wait up to one second for other CPUs to stop */
+ timeout = USEC_PER_SEC;
+ while (num_online_cpus() > 1 && timeout--)
+ udelay(1);
+
+ if (num_online_cpus() > 1)
+ pr_warn("SMP: failed to stop secondary CPUs\n");
}
void smp_send_reschedule(int cpu)
--
2.19.2
--
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] riscv: don't stop itself in smp_send_stop
2018-11-29 16:32 [PATCH] riscv: don't stop itself in smp_send_stop Andreas Schwab
@ 2018-11-29 18:52 ` Atish Patra
2018-12-10 10:58 ` Andreas Schwab
0 siblings, 1 reply; 7+ messages in thread
From: Atish Patra @ 2018-11-29 18:52 UTC (permalink / raw)
To: Andreas Schwab, linux-riscv
On 11/29/18 8:32 AM, Andreas Schwab wrote:
> Only run ipi_stop on other cpus, not itself. Don't wait for ipi_stop to
> return, since it never does. Also, mark each cpu offline.
>
> Signed-off-by: Andreas Schwab <schwab@suse.de>
> ---
> arch/riscv/kernel/smp.c | 17 ++++++++++++++++-
> 1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index 57b1383e5e..5c6bb5fe5a 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -23,6 +23,7 @@
> #include <linux/smp.h>
> #include <linux/sched.h>
> #include <linux/seq_file.h>
> +#include <linux/delay.h>
>
> #include <asm/sbi.h>
> #include <asm/tlbflush.h>
> @@ -148,13 +149,27 @@ void arch_send_call_function_single_ipi(int cpu)
>
> static void ipi_stop(void *unused)
> {
> + set_cpu_online (smp_processor_id(), false);
> while (1)
> wait_for_interrupt();
> }
>
> void smp_send_stop(void)
> {
> - on_each_cpu(ipi_stop, NULL, 1);
> + unsigned long timeout;
> + struct cpumask mask;
> +
No need to do mask copy this, if num_online_cpus() == 1.
> + cpumask_copy (&mask, cpu_online_mask);
> + cpumask_clear_cpu (smp_processor_id (), &mask);
> + on_each_cpu_mask(&mask, ipi_stop, NULL, 0);
The function header of on_each_cpu_mask says func should be
non-blocking. Not sure if that's a hard requirement or not.
If it is, we can introduce an extra IPI_STOP in ipi_names and just send
ipi. The receiving CPU can handle the IPI_STOP.
> +
> + /* Wait up to one second for other CPUs to stop */
> + timeout = USEC_PER_SEC;
> + while (num_online_cpus() > 1 && timeout--)
> + udelay(1);
> +
> + if (num_online_cpus() > 1)
> + pr_warn("SMP: failed to stop secondary CPUs\n");
May be print the remaining online cpus so that we know which one failed ?
Regards,
Atish
> }
>
> void smp_send_reschedule(int cpu)
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] riscv: don't stop itself in smp_send_stop
2018-11-29 18:52 ` Atish Patra
@ 2018-12-10 10:58 ` Andreas Schwab
2018-12-11 0:06 ` Atish Patra
2018-12-11 10:20 ` [PATCH v2] " Andreas Schwab
0 siblings, 2 replies; 7+ messages in thread
From: Andreas Schwab @ 2018-12-10 10:58 UTC (permalink / raw)
To: Atish Patra; +Cc: linux-riscv
On Nov 29 2018, Atish Patra <atish.patra@wdc.com> wrote:
>> + cpumask_copy (&mask, cpu_online_mask);
>> + cpumask_clear_cpu (smp_processor_id (), &mask);
>> + on_each_cpu_mask(&mask, ipi_stop, NULL, 0);
>
> The function header of on_each_cpu_mask says func should be
> non-blocking. Not sure if that's a hard requirement or not.
What does that mean?
>> +
>> + /* Wait up to one second for other CPUs to stop */
>> + timeout = USEC_PER_SEC;
>> + while (num_online_cpus() > 1 && timeout--)
>> + udelay(1);
>> +
>> + if (num_online_cpus() > 1)
>> + pr_warn("SMP: failed to stop secondary CPUs\n");
>
> May be print the remaining online cpus so that we know which one failed ?
I have copied that from arm. The only user of smp_send_stop is panic,
which can cope with that, and the probability of that happening is very
low anyway.
Andreas.
--
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] riscv: don't stop itself in smp_send_stop
2018-12-10 10:58 ` Andreas Schwab
@ 2018-12-11 0:06 ` Atish Patra
2018-12-11 10:20 ` [PATCH v2] " Andreas Schwab
1 sibling, 0 replies; 7+ messages in thread
From: Atish Patra @ 2018-12-11 0:06 UTC (permalink / raw)
To: Andreas Schwab; +Cc: linux-riscv
On 12/10/18 2:58 AM, Andreas Schwab wrote:
> On Nov 29 2018, Atish Patra <atish.patra@wdc.com> wrote:
>
>>> + cpumask_copy (&mask, cpu_online_mask);
>>> + cpumask_clear_cpu (smp_processor_id (), &mask);
>>> + on_each_cpu_mask(&mask, ipi_stop, NULL, 0);
>>
>> The function header of on_each_cpu_mask says func should be
>> non-blocking. Not sure if that's a hard requirement or not.
>
> What does that mean?
>
https://elixir.bootlin.com/linux/v4.20-rc6/source/kernel/smp.c#L617
>>> +
>>> + /* Wait up to one second for other CPUs to stop */
>>> + timeout = USEC_PER_SEC;
>>> + while (num_online_cpus() > 1 && timeout--)
>>> + udelay(1);
>>> +
>>> + if (num_online_cpus() > 1)
>>> + pr_warn("SMP: failed to stop secondary CPUs\n");
>>
>> May be print the remaining online cpus so that we know which one failed ?
>
> I have copied that from arm. The only user of smp_send_stop is panic,
> which can cope with that, and the probability of that happening is very
> low anyway.
>
My suggestion was based on arm64 code. IMHO, it makes more sense to
follow arm64 code than arm.
https://elixir.bootlin.com/linux/v4.20-rc6/source/arch/arm64/kernel/smp.c#L955
Regards,
Atish
> Andreas.
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2] riscv: don't stop itself in smp_send_stop
2018-12-10 10:58 ` Andreas Schwab
2018-12-11 0:06 ` Atish Patra
@ 2018-12-11 10:20 ` Andreas Schwab
2018-12-13 23:27 ` Atish Patra
1 sibling, 1 reply; 7+ messages in thread
From: Andreas Schwab @ 2018-12-11 10:20 UTC (permalink / raw)
To: Atish Patra; +Cc: linux-riscv
Add IPI_CPU_STOP message and use it in smp_send_stop to stop other cpus,
but not itself. Mark cpu offline on reception of IPI_CPU_STOP.
Signed-off-by: Andreas Schwab <schwab@suse.de>
---
arch/riscv/kernel/smp.c | 43 ++++++++++++++++++++++++++++++++++-------
1 file changed, 36 insertions(+), 7 deletions(-)
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index 57b1383e5e..b813264416 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -23,6 +23,7 @@
#include <linux/smp.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
+#include <linux/delay.h>
#include <asm/sbi.h>
#include <asm/tlbflush.h>
@@ -31,6 +32,7 @@
enum ipi_message_type {
IPI_RESCHEDULE,
IPI_CALL_FUNC,
+ IPI_CPU_STOP,
IPI_MAX
};
@@ -66,6 +68,13 @@ int setup_profiling_timer(unsigned int multiplier)
return -EINVAL;
}
+static void ipi_stop(void)
+{
+ set_cpu_online (smp_processor_id(), false);
+ while (1)
+ wait_for_interrupt();
+}
+
void riscv_software_interrupt(void)
{
unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
@@ -94,6 +103,11 @@ void riscv_software_interrupt(void)
generic_smp_call_function_interrupt();
}
+ if (ops & (1 << IPI_CPU_STOP)) {
+ stats[IPI_CPU_STOP]++;
+ ipi_stop();
+ }
+
BUG_ON((ops >> IPI_MAX) != 0);
/* Order data access and bit testing. */
@@ -121,6 +135,7 @@ send_ipi_message(const struct cpumask *to_whom, enum ipi_message_type operation)
static const char * const ipi_names[] = {
[IPI_RESCHEDULE] = "Rescheduling interrupts",
[IPI_CALL_FUNC] = "Function call interrupts",
+ [IPI_CPU_STOP] = "CPU stop interrupts",
};
void show_ipi_stats(struct seq_file *p, int prec)
@@ -146,15 +161,29 @@ void arch_send_call_function_single_ipi(int cpu)
send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
}
-static void ipi_stop(void *unused)
-{
- while (1)
- wait_for_interrupt();
-}
-
void smp_send_stop(void)
{
- on_each_cpu(ipi_stop, NULL, 1);
+ unsigned long timeout;
+
+ if (num_online_cpus() > 1) {
+ cpumask_t mask;
+
+ cpumask_copy(&mask, cpu_online_mask);
+ cpumask_clear_cpu(smp_processor_id(), &mask);
+
+ if (system_state <= SYSTEM_RUNNING)
+ pr_crit("SMP: stopping secondary CPUs\n");
+ send_ipi_message(&mask, IPI_CPU_STOP);
+ }
+
+ /* Wait up to one second for other CPUs to stop */
+ timeout = USEC_PER_SEC;
+ while (num_online_cpus() > 1 && timeout--)
+ udelay(1);
+
+ if (num_online_cpus() > 1)
+ pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
+ cpumask_pr_args(cpu_online_mask));
}
void smp_send_reschedule(int cpu)
--
2.20.0
--
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2] riscv: don't stop itself in smp_send_stop
2018-12-11 10:20 ` [PATCH v2] " Andreas Schwab
@ 2018-12-13 23:27 ` Atish Patra
2019-01-03 19:22 ` Palmer Dabbelt
0 siblings, 1 reply; 7+ messages in thread
From: Atish Patra @ 2018-12-13 23:27 UTC (permalink / raw)
To: Andreas Schwab; +Cc: linux-riscv
On 12/11/18 2:20 AM, Andreas Schwab wrote:
> Add IPI_CPU_STOP message and use it in smp_send_stop to stop other cpus,
> but not itself. Mark cpu offline on reception of IPI_CPU_STOP.
>
> Signed-off-by: Andreas Schwab <schwab@suse.de>
> ---
> arch/riscv/kernel/smp.c | 43 ++++++++++++++++++++++++++++++++++-------
> 1 file changed, 36 insertions(+), 7 deletions(-)
>
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index 57b1383e5e..b813264416 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -23,6 +23,7 @@
> #include <linux/smp.h>
> #include <linux/sched.h>
> #include <linux/seq_file.h>
> +#include <linux/delay.h>
>
> #include <asm/sbi.h>
> #include <asm/tlbflush.h>
> @@ -31,6 +32,7 @@
> enum ipi_message_type {
> IPI_RESCHEDULE,
> IPI_CALL_FUNC,
> + IPI_CPU_STOP,
> IPI_MAX
> };
>
> @@ -66,6 +68,13 @@ int setup_profiling_timer(unsigned int multiplier)
> return -EINVAL;
> }
>
> +static void ipi_stop(void)
> +{
> + set_cpu_online (smp_processor_id(), false);
> + while (1)
> + wait_for_interrupt();
> +}
> +
> void riscv_software_interrupt(void)
> {
> unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
> @@ -94,6 +103,11 @@ void riscv_software_interrupt(void)
> generic_smp_call_function_interrupt();
> }
>
> + if (ops & (1 << IPI_CPU_STOP)) {
> + stats[IPI_CPU_STOP]++;
> + ipi_stop();
> + }
> +
> BUG_ON((ops >> IPI_MAX) != 0);
>
> /* Order data access and bit testing. */
> @@ -121,6 +135,7 @@ send_ipi_message(const struct cpumask *to_whom, enum ipi_message_type operation)
> static const char * const ipi_names[] = {
> [IPI_RESCHEDULE] = "Rescheduling interrupts",
> [IPI_CALL_FUNC] = "Function call interrupts",
> + [IPI_CPU_STOP] = "CPU stop interrupts",
> };
>
> void show_ipi_stats(struct seq_file *p, int prec)
> @@ -146,15 +161,29 @@ void arch_send_call_function_single_ipi(int cpu)
> send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
> }
>
> -static void ipi_stop(void *unused)
> -{
> - while (1)
> - wait_for_interrupt();
> -}
> -
> void smp_send_stop(void)
> {
> - on_each_cpu(ipi_stop, NULL, 1);
> + unsigned long timeout;
> +
> + if (num_online_cpus() > 1) {
> + cpumask_t mask;
> +
> + cpumask_copy(&mask, cpu_online_mask);
> + cpumask_clear_cpu(smp_processor_id(), &mask);
> +
> + if (system_state <= SYSTEM_RUNNING)
> + pr_crit("SMP: stopping secondary CPUs\n");
> + send_ipi_message(&mask, IPI_CPU_STOP);
> + }
> +
> + /* Wait up to one second for other CPUs to stop */
> + timeout = USEC_PER_SEC;
> + while (num_online_cpus() > 1 && timeout--)
> + udelay(1);
> +
> + if (num_online_cpus() > 1)
> + pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
> + cpumask_pr_args(cpu_online_mask));
> }
>
> void smp_send_reschedule(int cpu)
>
Checkpatch warnings.
WARNING: space prohibited between function name and open parenthesis '('
#40: FILE: arch/riscv/kernel/smp.c:73:
+ set_cpu_online (smp_processor_id(), false);
WARNING: Prefer pr_warn(... to pr_warning(...
#100: FILE: arch/riscv/kernel/smp.c:185:
+ pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
Otherwise, Looks good to me.
Reviewed-by: Atish Patra <atish.patra@wdc.com>
Regards,
Atish
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] riscv: don't stop itself in smp_send_stop
2018-12-13 23:27 ` Atish Patra
@ 2019-01-03 19:22 ` Palmer Dabbelt
0 siblings, 0 replies; 7+ messages in thread
From: Palmer Dabbelt @ 2019-01-03 19:22 UTC (permalink / raw)
To: atish.patra; +Cc: schwab, linux-riscv
On Thu, 13 Dec 2018 15:27:36 PST (-0800), atish.patra@wdc.com wrote:
> On 12/11/18 2:20 AM, Andreas Schwab wrote:
>> Add IPI_CPU_STOP message and use it in smp_send_stop to stop other cpus,
>> but not itself. Mark cpu offline on reception of IPI_CPU_STOP.
>>
>> Signed-off-by: Andreas Schwab <schwab@suse.de>
>> ---
>> arch/riscv/kernel/smp.c | 43 ++++++++++++++++++++++++++++++++++-------
>> 1 file changed, 36 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
>> index 57b1383e5e..b813264416 100644
>> --- a/arch/riscv/kernel/smp.c
>> +++ b/arch/riscv/kernel/smp.c
>> @@ -23,6 +23,7 @@
>> #include <linux/smp.h>
>> #include <linux/sched.h>
>> #include <linux/seq_file.h>
>> +#include <linux/delay.h>
>>
>> #include <asm/sbi.h>
>> #include <asm/tlbflush.h>
>> @@ -31,6 +32,7 @@
>> enum ipi_message_type {
>> IPI_RESCHEDULE,
>> IPI_CALL_FUNC,
>> + IPI_CPU_STOP,
>> IPI_MAX
>> };
>>
>> @@ -66,6 +68,13 @@ int setup_profiling_timer(unsigned int multiplier)
>> return -EINVAL;
>> }
>>
>> +static void ipi_stop(void)
>> +{
>> + set_cpu_online (smp_processor_id(), false);
>> + while (1)
>> + wait_for_interrupt();
>> +}
>> +
>> void riscv_software_interrupt(void)
>> {
>> unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
>> @@ -94,6 +103,11 @@ void riscv_software_interrupt(void)
>> generic_smp_call_function_interrupt();
>> }
>>
>> + if (ops & (1 << IPI_CPU_STOP)) {
>> + stats[IPI_CPU_STOP]++;
>> + ipi_stop();
>> + }
>> +
>> BUG_ON((ops >> IPI_MAX) != 0);
>>
>> /* Order data access and bit testing. */
>> @@ -121,6 +135,7 @@ send_ipi_message(const struct cpumask *to_whom, enum ipi_message_type operation)
>> static const char * const ipi_names[] = {
>> [IPI_RESCHEDULE] = "Rescheduling interrupts",
>> [IPI_CALL_FUNC] = "Function call interrupts",
>> + [IPI_CPU_STOP] = "CPU stop interrupts",
>> };
>>
>> void show_ipi_stats(struct seq_file *p, int prec)
>> @@ -146,15 +161,29 @@ void arch_send_call_function_single_ipi(int cpu)
>> send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC);
>> }
>>
>> -static void ipi_stop(void *unused)
>> -{
>> - while (1)
>> - wait_for_interrupt();
>> -}
>> -
>> void smp_send_stop(void)
>> {
>> - on_each_cpu(ipi_stop, NULL, 1);
>> + unsigned long timeout;
>> +
>> + if (num_online_cpus() > 1) {
>> + cpumask_t mask;
>> +
>> + cpumask_copy(&mask, cpu_online_mask);
>> + cpumask_clear_cpu(smp_processor_id(), &mask);
>> +
>> + if (system_state <= SYSTEM_RUNNING)
>> + pr_crit("SMP: stopping secondary CPUs\n");
>> + send_ipi_message(&mask, IPI_CPU_STOP);
>> + }
>> +
>> + /* Wait up to one second for other CPUs to stop */
>> + timeout = USEC_PER_SEC;
>> + while (num_online_cpus() > 1 && timeout--)
>> + udelay(1);
>> +
>> + if (num_online_cpus() > 1)
>> + pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
>> + cpumask_pr_args(cpu_online_mask));
>> }
>>
>> void smp_send_reschedule(int cpu)
>>
>
> Checkpatch warnings.
> WARNING: space prohibited between function name and open parenthesis '('
> #40: FILE: arch/riscv/kernel/smp.c:73:
> + set_cpu_online (smp_processor_id(), false);
>
> WARNING: Prefer pr_warn(... to pr_warning(...
> #100: FILE: arch/riscv/kernel/smp.c:185:
> + pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
>
>
> Otherwise, Looks good to me.
> Reviewed-by: Atish Patra <atish.patra@wdc.com>
I've fixed these up and added the patch to my fix queue, it should go up next
week.
Thanks!
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-01-03 19:22 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-29 16:32 [PATCH] riscv: don't stop itself in smp_send_stop Andreas Schwab
2018-11-29 18:52 ` Atish Patra
2018-12-10 10:58 ` Andreas Schwab
2018-12-11 0:06 ` Atish Patra
2018-12-11 10:20 ` [PATCH v2] " Andreas Schwab
2018-12-13 23:27 ` Atish Patra
2019-01-03 19:22 ` Palmer Dabbelt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).