Re: [PATCH v2 1/2] sched/wait: Add add_wait_queue_priority()

From: David Woodhouse <dwmw2@infradead.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: linux-kernel@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
	Daniel Bristot de Oliveira <bristot@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	kvm@vger.kernel.org, Oleg Nesterov <oleg@redhat.com>
Subject: Re: [PATCH v2 1/2] sched/wait: Add add_wait_queue_priority()
Date: Tue, 27 Oct 2020 20:49:45 +0000	[thread overview]
Message-ID: <1e10666c086c0f5221d399de974fcff5f05e61b4.camel@infradead.org> (raw)
In-Reply-To: <20201027203041.GS2628@hirez.programming.kicks-ass.net>

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Tue, 2020-10-27 at 21:30 +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2020 at 07:27:59PM +0000, David Woodhouse wrote:
> 
> > > While looking at this I found that weird __add_wait_queue_exclusive()
> > > which is used by fs/eventpoll.c and does something similar, except it
> > > doesn't keep the FIFO order.
> > 
> > It does, doesn't it? Except those so-called "exclusive" entries end up
> > in FIFO order amongst themselves at the *tail* of the queue, to be
> > woken up only after all the other entries before them *haven't* been
> > excluded.
> 
> __add_wait_queue_exclusive() uses __add_wait_queue() which does
> list_add(). It does _not_ add at the tail like normal exclusive users,
> and there is exactly _1_ user in tree that does this.
> 
> I'm not exactly sure how this happened, but:
> 
>   add_wait_queue_exclusive()
> 
> and
> 
>   __add_wait_queue_exclusive()
> 
> are not related :-(

Oh, that is *particularly* special.

It sounds like the __add_wait_queue_exclusive() version is a half-baked 
attempt at doing what I'm doing here, except....

> > > The Changelog doesn't state how important this property is to you.
> > 
> > Because it isn't :)
> > 
> > The ordering is:
> > 
> >  { PRIORITY }*  { NON-EXCLUSIVE }* { EXCLUSIVE(sic) }*
> > 
> > I care that PRIORITY comes before the others, because I want to
> > actually exclude the others. Especially the "non-exclusive" ones, which
> > the 'exclusive' ones don't actually exclude.
> > 
> > I absolutely don't care about ordering *within* the set of PRIORITY
> > entries, since as I said I expect there to be only one.
> 
> Then you could arguably do something like:
> 
> 	spin_lock_irqsave(&wq_head->lock, flags);
> 	__add_wait_queue_exclusive(wq_head, wq_entry);
> 	spin_unlock_irqrestore(&wq_head->lock, flags);
> 
> and leave it at that.

.. the problem with that is that other waiters *can* end up on the
queue before it, if they are added later. I don't know if the existing
user (ep_poll) cares, but I do.

> But now I'm itching to fix that horrible naming... tomorrow perhaps.

:)

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5174 bytes --]