Re: [Question]: try to fix contention between expire_timers and try_to_del_timer_sync

From: Will Deacon <will.deacon@arm.com>
To: Vikram Mulukutla <markivx@codeaurora.org>
Cc: qiaozhou <qiaozhou@asrmicro.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	John Stultz <john.stultz@linaro.org>,
	sboyd@codeaurora.org, LKML <linux-kernel@vger.kernel.org>,
	Wang Wilbur <wilburwang@asrmicro.com>,
	Marc Zyngier <marc.zyngier@arm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-kernel-owner@vger.kernel.org, sudeep.holla@arm.com
Subject: Re: [Question]: try to fix contention between expire_timers and try_to_del_timer_sync
Date: Mon, 31 Jul 2017 14:13:21 +0100	[thread overview]
Message-ID: <20170731131321.GB1737@arm.com> (raw)
In-Reply-To: <2aa9684cf9c889ee9fdc8550b4388af6@codeaurora.org>

Hi Vikram,

On Fri, Jul 28, 2017 at 12:09:38PM -0700, Vikram Mulukutla wrote:
> On 2017-07-28 02:28, Will Deacon wrote:
> >On Thu, Jul 27, 2017 at 06:10:34PM -0700, Vikram Mulukutla wrote:
> 
> <snip>
> 
> >>
> >>I think we should have this discussion now - I brought this up earlier
> >>[1]
> >>and I promised a test case that I completely forgot about - but here it
> >>is (attached). Essentially a Big CPU in an acquire-check-release loop
> >>will have an unfair advantage over a little CPU concurrently attempting
> >>to acquire the same lock, in spite of the ticket implementation. If the
> >>Big
> >>CPU needs the little CPU to make forward progress : livelock.
> >>
> 
> <snip>
> 
> >>
> >>One solution was to use udelay(1) in such loops instead of cpu_relax(),
> >>but
> >>that's not very 'relaxing'. I'm not sure if there's something we could
> >>do
> >>within the ticket spin-lock implementation to deal with this.
> >
> >Does bodging cpu_relax to back-off to wfe after a while help? The event
> >stream will wake it up if nothing else does. Nasty patch below, but I'd be
> >interested to know whether or not it helps.
> >
> >Will
> >
> This does seem to help. Here's some data after 5 runs with and without the
> patch.

Blimey, that does seem to make a difference. Shame it's so ugly! Would you
be able to experiment with other values for CPU_RELAX_WFE_THRESHOLD? I had
it set to 10000 in the diff I posted, but that might be higher than optimal.
It would be interested to see if it correlates with num_possible_cpus()
for the highly contended case.

Will