linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Peter Zijlstra <peterz@infradead.org>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Clark Williams <williams@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Nick Piggin <nickpiggin@yahoo.com.au>
Subject: Re: [PATCH] sched: Do not release current rq lock on non contended double_lock_balance()
Date: Tue, 14 Jun 2016 21:42:17 +0200	[thread overview]
Message-ID: <20160614194217.GK30921@twins.programming.kicks-ass.net> (raw)
In-Reply-To: <20160614140228.0ecf15af@grimm.local.home>

On Tue, Jun 14, 2016 at 02:02:28PM -0400, Steven Rostedt wrote:
> On Tue, 14 Jun 2016 13:58:20 +0200
> Peter Zijlstra <peterz@infradead.org> wrote:
> > And it does indeed make the hold time harder to analyze.
> > 
> > For instance; pull_rt_task() does:
> > 
> > 	for_each_cpu() {
> > 		double_lock_balance(this, that);
> > 		...
> > 		double_unlock_balance(this, that);
> > 	}
> > 
> > Which, with the trylock, ends up with a max possible hold time of
> > O(nr_cpus).
> 
> Sure, but I think we should try to limit that loop too, because that
> loop itself is what is triggering the large latency for me, because
> it constantly releases a spinlock and has to wait. This loop is done
> with preemption disabled.

Much worse, its done with IRQs disabled. But that affects only the local
CPU. Holding the lock that long affects all other CPUs too.

> > Unlikely, sure, but RT is a game of upper bounds etc.
> 
> Sure, but should we force worst case all the time?

How is that relevant? Either you have a bounded operation or you don't.

> We do a lot of optimization to allow for good throughput as well.

Only within keeping the upper bounds. The moment you let go of that,
you've destroyed RT.

> > So should we maybe do something like:
> > 
> > 	if (unlikely(raw_spin_is_contended(&this_rq->lock) ||
> > 	             !raw_spin_trylock(&busiest->lock))) {
> 
> Why do we care if this_rq is contended? 

To bound hold time.

> That's exactly what causes
> large latency to happen. Because when we let go of this_rq, this fast
> path becomes much slower because now it must wait for whatever is
> waiting on it to finish. The more CPUs you have, the bigger this issue
> becomes.

Yes, icky issue.

And while the numbers look pretty I'm not sure you've not introduced
another, less likely, issue.

  reply	other threads:[~2016-06-14 19:42 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-06-13 16:37 Steven Rostedt
2016-06-14 11:58 ` Peter Zijlstra
2016-06-14 17:52   ` Steven Rostedt
2016-06-14 18:02   ` Steven Rostedt
2016-06-14 19:42     ` Peter Zijlstra [this message]
2016-06-15 11:14     ` Peter Zijlstra
2016-06-15 16:13       ` Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160614194217.GK30921@twins.programming.kicks-ass.net \
    --to=peterz@infradead.org \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=nickpiggin@yahoo.com.au \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --cc=williams@redhat.com \
    --subject='Re: [PATCH] sched: Do not release current rq lock on non contended double_lock_balance()' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).