Re: [PATCH 1/2] sched/wait: Break up long wake list walk

From: Andi Kleen <ak@linux.intel.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Liang, Kan" <kan.liang@intel.com>,
	Mel Gorman <mgorman@techsingularity.net>,
	Mel Gorman <mgorman@suse.de>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Tim Chen <tim.c.chen@linux.intel.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@elte.hu>,
	Andrew Morton <akpm@linux-foundation.org>,
	Johannes Weiner <hannes@cmpxchg.org>, Jan Kara <jack@suse.cz>,
	linux-mm <linux-mm@kvack.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 1/2] sched/wait: Break up long wake list walk
Date: Fri, 18 Aug 2017 13:05:10 -0700	[thread overview]
Message-ID: <20170818200510.GQ28715@tassilo.jf.intel.com> (raw)
In-Reply-To: <CA+55aFxZjjqUM4kPvNEeZahPovBHFATiwADj-iPTDN0-jnU67Q@mail.gmail.com>

> I was really hoping that we'd root-cause this and have a solution (and
> then apply Tim's patch as a "belt and suspenders" kind of thing), but

One thing I wanted to point out is that Tim's patch seems to make
several schedule intensive micro benchmarks faster. 

I think what's happening is that it allows more parallelism during wakeup:

Normally it's like

CPU 1                           CPU 2               CPU 3                    .....

LOCK
wake up tasks on other CPUs    woken up             woken up
UNLOCK                         SPIN on waitq lock   SPIN on waitq lock
                               LOCK
                               remove waitq
                               UNLOCk
                                                    LOCK
                                                    remove waitq
                                                    UNLOCK

So everything is serialized.

But with the bookmark patch the other CPUs can go through the "remove waitq" sequence
earlier because they have a chance to get a go at the lock and do it in parallel 
with the main wakeup.  

Tim used a 64 task threshold for the bookmark. That may be actually too large.
It may even be faster to use a shorter one.

So I think it's more than a bandaid, but likely a useful performance improvement
even for less extreme wait queues.

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>