Re: [RFC-PROTOTYPE 1/1] mm: Add __GFP_FAST_TRY flag

From: Vlastimil Babka <vbabka@suse.cz>
To: Matthew Wilcox <willy@infradead.org>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>, RCU <rcu@vger.kernel.org>,
	linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
	"Paul E . McKenney" <paulmck@kernel.org>,
	"Theodore Y . Ts'o" <tytso@mit.edu>,
	Joel Fernandes <joel@joelfernandes.org>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Oleksiy Avramchenko <oleksiy.avramchenko@sonymobile.com>
Subject: Re: [RFC-PROTOTYPE 1/1] mm: Add __GFP_FAST_TRY flag
Date: Tue, 4 Aug 2020 19:34:18 +0200	[thread overview]
Message-ID: <fe1dcc38-c968-0439-cb01-431cf004fc38@suse.cz> (raw)
In-Reply-To: <20200804171203.GH23808@casper.infradead.org>

On 8/4/20 7:12 PM, Matthew Wilcox wrote:
> On Tue, Aug 04, 2020 at 07:02:14PM +0200, Vlastimil Babka wrote:
>> > 2) There was a proposal from Matthew Wilcox: https://lkml.org/lkml/2020/7/31/1015
>> > 
>> > <snip>
>> > On non-RT, we could make that lock a raw spinlock.  On RT, we could
>> > decline to take the lock.  We'd need to abstract the spin_lock() away
>> > behind zone_lock(zone), but that should be OK.
>> > <snip>
>> > 
>> > It would be great to use any existing flag, say GFP_NOWAIT. Suppose we
>> > decline to take the lock across the page allocator for RT. But there is
>> > at least one path that does it outside of the page allocator. GFP_NOWAIT
>> > can wakeup the kswapd, whereas a "wake-up path" uses sleepable lock:
>> > 
>> > wakeup_kswapd() -> wake_up_interruptible(&pgdat->kswapd_wait).
>> > 
>> > Probably it can be fixed by the excluding of waking of the kswapd process
>> > defining something like below:
>> 
>> Is something missing here?
>> 
>> > what is equal to zero and i am not sure if __get_free_page(0) handles
>> > all that correctly, though it allocates and seems working on my test
>> > machine! Please note it is related to "if we can reuse existing flags".
>> > 
>> > In the meantime, please see below for a patch that adds a __GFP_FAST_TRY,
>> > which can at least serve as a baseline against which other proposals can
>> > be compared. The patch is based on the 5.8.0-rc3.
>> > 
>> > Please RFC.
>> 
>> At first glance __GFP_FAST_TRY (more descriptive name? __GFP_NO_LOCKS?) seems
>> better than doing weird things with GFP_NOWAIT, but depends on the real benefits
>> (hence my first questions).
> 
> I think what Vlad is trying to say is that even GFP_NOWAIT will wake
> kswapd, which involves taking a spinlock.  If you specify 0 in your GFP
> flags, then we won't wake kswapd.  So a simple:
> 
> #define GFP_NOLOCKS	0
> 
> should do the trick (modulo various casting, blah blah blah)

Ah, you're right, waking up kswapd is is only done with __GFP_KSWAPD_RECLAIM and
GFP_NOWAIT equals to that. So that's easy to avoid for the rcu allocation.

But still IIUC option 2) would mean that even with "#define GFP_NOLOCKS	0" would
mean we need to abstract away the zone lock, and behave differently depending on
the kernel being RT, and inadvertedly changing other users that happen to
specify gfp where "gfp & GFP_RECLAIM_MASK == 0" (or however we would exactly
check if we can take the lock on RT kernel). That sounds too complicated to me.