Re: Userfaultfd doesn't seem to break out of poll on fd close

[Andrea Arcangeli <aarcange@redhat.com>]

Hello everyone,

On Wed, Apr 15, 2020 at 11:25:46AM -0300, Jason Gunthorpe wrote:
>           CPU1                            CPU2                  CPU3
>  fds[i]->fd = userfaultfd;
>  while()
>                                        close(userfaultfd)
>                                        pthread_join()
>                                                             someother_fd = open()
>                                                             userfaultfd == someother_fd
>      poll(fds)   // <- Still sleeps
> 
> The kernel should not be trying to wake poll from fd release, and
> userspace should not close a FD that is currently under poll.
> 
> Besides, it really does look like poll holds the fget while doing its
> work (see poll_freewait), so fops release() won't be called anyhow..

Agreed, poll does fdget (not userfaultfd_poll) so there's no way
->release will be called when the fd is closed in the other thread.

The simple way to fix this is to implement a ->flush operation
(userfaultfd_flush), perhaps something like this would work (untested):

static int userfaultfd_flush(struct file *file, fl_owner_t id)
{
	struct userfaultfd_ctx *ctx = file->private_data;
	wake_up_poll(&ctx->fd_wqh, EPOLLHUP);
}

If eventfd and pipes all behave identical to uffd (they should as they
don't seem to implement flush) I'm not sure if there's good enough
justification to deviate from the default VFS behavior here.

The file flush operation is usually meaningful when the fd represent
data stored remotely, like with nfs, for uffd close() has no special
semantics.

With threads, you can get the wakeup by other means as Peter
suggested. Then you can close the uffd in the parent after poll
returns.

Alternatively if you want to rely on uffd to send the poll wakeup you
could use UFFDIO_WAKE instead of closing the fd, and still close the fd
after poll returns.

Overall the more normal thing to do is to close the uffd after poll
returns, if you can't do that (or if it's less efficient doing that)
it'd be interesting to know why to better evaluate this. By just
looking the testcase there's no way to tell if you gain something
meaningful by closing the fd during poll..

Thanks,
Andrea