From: Stafford Horne <shorne@gmail.com>
To: Matt Redfearn <matt.redfearn@mips.com>
Cc: LKML <linux-kernel@vger.kernel.org>,
Jonas Bonn <jonas@southpole.se>,
Stefan Kristiansson <stefan.kristiansson@saunalahti.fi>,
Jan Henrik Weinstock <jan.weinstock@ice.rwth-aachen.de>,
Matt Redfearn <matt.redfearn@imgtec.com>,
James Hogan <james.hogan@imgtec.com>,
Thomas Gleixner <tglx@linutronix.de>,
openrisc@lists.librecores.org,
Matija Glavinic Pecotic <matija.glavinic-pecotic.ext@nokia.com>
Subject: Re: [PATCH v4 13/13] openrisc: add tick timer multi-core sync logic
Date: Wed, 1 Nov 2017 09:34:47 +0900 [thread overview]
Message-ID: <20171101003447.GC29237@lianli.shorne-pla.net> (raw)
In-Reply-To: <20171031231759.GB29237@lianli.shorne-pla.net>
On Wed, Nov 01, 2017 at 08:17:59AM +0900, Stafford Horne wrote:
> On Tue, Oct 31, 2017 at 02:06:21PM +0000, Matt Redfearn wrote:
> > Hi,
> >
> >
> > On 29/10/17 23:11, Stafford Horne wrote:
> > > In case timers are not in sync when cpus start (i.e. hot plug / offset
> > > resets) we need to synchronize the secondary cpus internal timer with
> > > the main cpu. This is needed as in OpenRISC SMP there is only one
> > > clocksource registered which reads from the same ttcr register on each
> > > cpu.
> > >
> > > This synchronization routine heavily borrows from mips implementation that
> > > does something similar.
> [..]
> > > diff --git a/arch/openrisc/kernel/smp.c b/arch/openrisc/kernel/smp.c
> > > index 4763b8b9161e..4d80ce6fa045 100644
> > > --- a/arch/openrisc/kernel/smp.c
> > > +++ b/arch/openrisc/kernel/smp.c
> > > @@ -100,6 +100,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
> > > pr_crit("CPU%u: failed to start\n", cpu);
> > > return -EIO;
> > > }
> > > + synchronise_count_master(cpu);
> > > return 0;
> > > }
> > > @@ -129,6 +130,8 @@ asmlinkage __init void secondary_start_kernel(void)
> > > set_cpu_online(cpu, true);
> > > complete(&cpu_running);
> > > + synchronise_count_slave(cpu);
> > > +
> >
> >
> > Note that until 8f46cca1e6c06a058374816887059bcc017b382f, the MIPS timer
> > synchronization code contained the possibility of deadlock. If you mark a
> > CPU online before it goes into the synchronize loop, then the boot CPU can
> > schedule a different thread and send IPIs to all "online" CPUs. It gets
> > stuck waiting for the secondary to ack it's IPI, since this secondary CPU
> > has not enabled IRQs yet, and is stuck waiting for the master to synchronise
> > with it. The system then deadlocks.
> > Commit 8f46cca1e6c06a058374816887059bcc017b382f fixed this for MIPS and you
> > might want to similarly move the
> >
> > set_cpu_online(cpu, true);
> >
> > after counters are synchronized.
>
> Thank you for the heads up. I do remember having interim issues with the timer
> syncing but I havent seen it for a while. I think I fixed it by also moving
> synchronise_count_slave.
>
> Let me double check. Also, I see your patch 8f46cca1e6c06a0583748168 was merged
> last year?
Hello,
I should have read a bit more closely, definitely this could be an issue if the
boot cpu has other things running.
However, looking at mainline I can see the clock sync comes after set_cpu_online
again after this patch in mips.
6f542ebeaee0 MIPS: Fix race on setting and getting cpu_online_mask
Author: Matija Glavinic Pecotic <matija.glavinic-pecotic.ext@nokia.com>
Is this deadlock an issue in mips again?
-Stafford
next prev parent reply other threads:[~2017-11-01 0:34 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-29 23:11 [PATCH v4 00/13] OpenRISC SMP Support Stafford Horne
2017-10-29 23:11 ` [PATCH v4 01/13] openrisc: use shadow registers to save regs on exception Stafford Horne
2017-10-29 23:11 ` [PATCH v4 02/13] openrisc: add 1 and 2 byte cmpxchg support Stafford Horne
2017-10-29 23:11 ` [PATCH v4 03/13] openrisc: use qspinlocks and qrwlocks Stafford Horne
2017-10-29 23:11 ` [PATCH v4 04/13] dt-bindings: add openrisc to vendor prefixes list Stafford Horne
2017-10-29 23:11 ` [PATCH v4 05/13] irqchip: add initial support for ompic Stafford Horne
2017-10-30 2:29 ` Marc Zyngier
2017-10-30 4:18 ` Stafford Horne
2017-10-30 6:11 ` Marc Zyngier
2017-11-01 12:17 ` Stafford Horne
2017-10-29 23:11 ` [PATCH v4 06/13] openrisc: initial SMP support Stafford Horne
2017-10-29 23:11 ` [PATCH v4 07/13] openrisc: fix initial preempt state for secondary cpu tasks Stafford Horne
2017-10-29 23:11 ` [PATCH v4 08/13] openrisc: sleep instead of spin on secondary wait Stafford Horne
2017-10-29 23:11 ` [PATCH v4 09/13] openrisc: add cacheflush support to fix icache aliasing Stafford Horne
2017-10-29 23:11 ` [PATCH v4 10/13] openrisc: add simple_smp dts and defconfig for simulators Stafford Horne
2017-10-29 23:11 ` [PATCH v4 11/13] openrisc: support framepointers and STACKTRACE_SUPPORT Stafford Horne
2017-10-29 23:11 ` [PATCH v4 12/13] openrisc: enable LOCKDEP_SUPPORT and irqflags tracing Stafford Horne
2017-10-29 23:11 ` [PATCH v4 13/13] openrisc: add tick timer multi-core sync logic Stafford Horne
2017-10-31 14:06 ` Matt Redfearn
2017-10-31 23:17 ` Stafford Horne
2017-11-01 0:34 ` Stafford Horne [this message]
2017-11-01 9:26 ` Matt Redfearn
2017-11-01 12:15 ` Stafford Horne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171101003447.GC29237@lianli.shorne-pla.net \
--to=shorne@gmail.com \
--cc=james.hogan@imgtec.com \
--cc=jan.weinstock@ice.rwth-aachen.de \
--cc=jonas@southpole.se \
--cc=linux-kernel@vger.kernel.org \
--cc=matija.glavinic-pecotic.ext@nokia.com \
--cc=matt.redfearn@imgtec.com \
--cc=matt.redfearn@mips.com \
--cc=openrisc@lists.librecores.org \
--cc=stefan.kristiansson@saunalahti.fi \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).