On Tue, 2020-10-27 at 21:30 +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2020 at 07:27:59PM +0000, David Woodhouse wrote:
> 
> > > While looking at this I found that weird __add_wait_queue_exclusive()
> > > which is used by fs/eventpoll.c and does something similar, except it
> > > doesn't keep the FIFO order.
> > 
> > It does, doesn't it? Except those so-called "exclusive" entries end up
> > in FIFO order amongst themselves at the *tail* of the queue, to be
> > woken up only after all the other entries before them *haven't* been
> > excluded.
> 
> __add_wait_queue_exclusive() uses __add_wait_queue() which does
> list_add(). It does _not_ add at the tail like normal exclusive users,
> and there is exactly _1_ user in tree that does this.
> 
> I'm not exactly sure how this happened, but:
> 
>   add_wait_queue_exclusive()
> 
> and
> 
>   __add_wait_queue_exclusive()
> 
> are not related :-(

I think that goes all the way back to here:

https://lkml.org/lkml/2007/5/4/530

It was rounded up in commit d47de16c72and subsequently "cleaned up"
into an inline in wait.h, but I don't think there was ever a reason for
it to be added to the head of the list instead of the tail.

So I think we can reasonably make __add_wait_queue_exclusive() do
precisely the same thing as add_wait_queue_exclusive() does (modulo
locking).

And then potentially rename them both to something that isn't quite
such a lie. And give me the one I want that *does* actually exclude
other waiters :)