Userfaultfd doesn't seem to break out of poll on fd close

From: Brian Geffon <bgeffon@google.com>
To: Andrea Arcangeli <aarcange@redhat.com>
Cc: linux-mm <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Sonny Rao <sonnyrao@google.com>, Peter Xu <peterx@redhat.com>
Subject: Userfaultfd doesn't seem to break out of poll on fd close
Date: Sun, 12 Apr 2020 13:10:40 -0700	[thread overview]
Message-ID: <CADyq12wPW69ovpW4akDY5PGBbrvnwsLO86=sSKTU4CB3dNwG3Q@mail.gmail.com> (raw)

Hi,
It seems that userfaultfd isn't woken from a poll when the file
descriptor is closed. It seems that it should be from the code in
userfault_ctx_release, but it appears that's not actually called
immediately. I have a simple standalone example that shows this
behavior. It's straight forward: one thread creates a userfaultfd and
then closes it after a second thread has entered a poll syscall, some
abbreviated strace output is below showing this and the code can be
seen here: https://gist.github.com/bgaff/9a8fbbe8af79c0e18502430d416df77e

Given that it's probably very common to have a dedicated thread remain
blocked indefinitely in a poll(2) waiting for faults there must be a
way to break it out early when it's closed. Am I missing something?

Thanks,
Brian

// Open a userfaultfd from Thread 1.
12:55:27.611942 userfaultfd(O_NONBLOCK) = 3
12:55:27.612007 ioctl(3, UFFDIO_API, {api=0xaa, features=0 =>
features=UFFD_FEATURE_EVENT_FORK|UFFD_FEATURE_EVENT_REMAP|UFFD_FEATURE_EVENT_REMOVE|UFFD_FEATURE_MISSING_HUGETLBFS|UFFD_FEATURE_MISSING_SHMEM|UFFD_FEATURE_EVENT_UNMAP|UFFD_FEATURE_SIGBUS|UFFD_FEATURE_THREAD_ID,
ioctls=1<<_UFFDIO_REGISTER|1<<_UFFDIO_UNREGISTER|1<<_UFFDIO_API}) = 0

// Create a second thread (Thread 2) to poll.
12:55:27.612447 clone(strace: Process 72730 attached
child_stack=0x7f30efa9ffb0,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0x7f30efaa09d0, tls=0x7f30efaa0700,
child_tidptr=0x7f30efaa09d0) = 72730

// Thread 2 will poll on the userfaultfd for up to 2000ms.
[pid 72730] 12:55:27.612676 poll([{fd=3, events=POLLIN}], 1, 2000
<unfinished ...>

// Thread 1 closes the userfaultfd and fcntl confirms it's closed:
[pid 72729] 12:55:28.612945 close(3)    = 0
[pid 72729] 12:55:28.613039 fcntl(3, F_GETFD) = -1 EBADF (Bad file descriptor)

// Poll technically times out and while loop back in do_poll it gets a POLLNVAL.
[pid 72730] 12:55:29.614906 <... poll resumed> ) = 1 ([{fd=3,
revents=POLLNVAL}])