Re: [PATCH] mm: filemap: Avoid unnecessary barries and waitqueue lookups in unlock_page fastpath v4

From: Oleg Nesterov <oleg@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Mel Gorman <mgorman@suse.de>,
	Andrew Morton <akpm@linux-foundation.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Vlastimil Babka <vbabka@suse.cz>, Jan Kara <jack@suse.cz>,
	Michal Hocko <mhocko@suse.cz>, Hugh Dickins <hughd@google.com>,
	Dave Hansen <dave.hansen@intel.com>,
	Linux Kernel <linux-kernel@vger.kernel.org>,
	Linux-MM <linux-mm@kvack.org>,
	Linux-FSDevel <linux-fsdevel@vger.kernel.org>,
	Paul McKenney <paulmck@linux.vnet.ibm.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	David Howells <dhowells@redhat.com>
Subject: Re: [PATCH] mm: filemap: Avoid unnecessary barries and waitqueue lookups in unlock_page fastpath v4
Date: Thu, 15 May 2014 17:34:24 +0200	[thread overview]
Message-ID: <20140515153424.GB30668@redhat.com> (raw)
In-Reply-To: <20140515132058.GL30445@twins.programming.kicks-ass.net>

On 05/15, Peter Zijlstra wrote:
>
> So I suppose I'm failing to see the problem with something like:

Yeeees, I was thinking about something like this too ;)

> static inline void lock_page(struct page *page)
> {
> 	if (!trylock_page(page))
> 		__lock_page(page);
> }
>
> static inline void unlock_page(struct page *page)
> {
> 	clear_bit_unlock(&page->flags, PG_locked);
> 	if (PageWaiters(page))
> 		__unlock_page();
> }

but in this case we need mb() before PageWaiters(), I guess.

> void __lock_page(struct page *page)
> {
> 	struct wait_queue_head_t *wqh = page_waitqueue(page);
> 	DEFINE_WAIT_BIT(wait, &page->flags, PG_locked);
>
> 	spin_lock_irq(&wqh->lock);
> 	if (!PageWaiters(page))
> 		SetPageWaiters(page);
>
> 	wait.flags |= WQ_FLAG_EXCLUSIVE;
> 	preempt_disable();

why?

> 	do {
> 		if (list_empty(&wait->task_list))
> 			__add_wait_queue_tail(wqh, &wait);
>
> 		set_current_state(TASK_UNINTERRUPTIBLE);
>
> 		if (test_bit(wait.key.bit_nr, wait.key.flags)) {
> 			spin_unlock_irq(&wqh->lock);
> 			schedule_preempt_disabled();
> 			spin_lock_irq(&wqh->lock);

OK, probably to avoid the preemption before schedule(). Still can't
undestand why this makes sense, but in this case it would be better
to do disable/enable under "if (test_bit())" ?

Of course, this needs more work for lock_page_killable(), but this
should be simple.

Oleg.