Re: [PATCH RFC] hwspinlock: Don't take software spinlock before hwspinlock

[Ohad Ben-Cohen <ohad@wizery.com>]

Hi Lina,

On Mon, May 18, 2015 at 6:03 PM, Lina Iyer <lina.iyer@linaro.org> wrote:
> The lock in question is used differently than traditional locks across
> processors. This lock helps synchronizes context transition from
> non-secure to secure on the same processor.
>
> The usecase, goes like this. In cpuidle, any core can be the last core
> to power down. The last man also holds the responsibility of shutting
> down shared resources like caches etc. The way the power down of a core
> works is, there are some high level decisions made in Linux and these
> decisions (like to flush and invalidate caches) etc gets transferred
> over to the the secure layer. The secure layer executes the ARM WFI that
> powers down the cpu, but uses these decisions passed into to determine
> if the cache needs to be invalidated upon wakeup etc.
>
> There is a possible race condition between what Linux thinks is the last
> core, vs what secure layer thinks is the last core. Lets say, two cores
> c0, c1 are going down. c1 is the second last core to go down from Linux
> as such, will not carry information about shared resources when making
> the SCM call. c1  made the SCM call, but is stuck handling some FIQs. In
> the meanwhile c0, goes idle and since its the last core in Linux,
> figures out the state of the shared resources. c0 calls into SCM, and
> ends up powering down earlier than c1. Per secure layer, the last core
> to go down is c1 and the votes of the shared resources are considered
> from that core. Things like cache invalidation without flush may happen
> as a result of this inconsistency of last man view point.
>
> The way we have solved it, Linux acquires a hw spinlock for each core,
> when calling into SCM and the secure monitor releases the spinlock. At
> any given time, only one core can switch the context from Linux to
> secure for power down operations. This guarantees the last man is
> synchronized between both Linux and secure. Another core may be spinning
> waiting for hw mutex, but they all happen serialized. This mutex is held
> in an irq disable context in cpuidle.
>
> There may be another processor spining to wait on hw mutex, but there
> isnt much to do otherwise, because the only operation at this time while
> holding the lock is to call into SCM and that would unlock the mutex.

Just to make sure I understand, is this how your scenario is solved?

- c1 goes down
- c0 goes down, carries information about shared resources
- c1 takes HWLOCK and calls into SCM, stuck handling FIQs
- c0 wants to call into SCM but is waiting spinning on HWLOCK
- c1 completes handling FIQs, goes idle, HWLOCK is released by secure monitor
- c0 takes HWLOCK, calls into SCM, shared resources handled correctly,

HWLOCK in this example is a single shared hwspinlock accessible by c0,
c1 and secure monitor.

Thanks,
Ohad.