Re: userfaultfd: usability issue due to lack of UFFD events ordering

[David Hildenbrand <david@redhat.com>]

On 31.01.22 15:05, Mike Rapoport wrote:
> On Mon, Jan 31, 2022 at 11:48:27AM +0100, David Hildenbrand wrote:
>> On 31.01.22 11:42, Mike Rapoport wrote:
>>> Hi Nadav,
>>>
>>> On Sat, Jan 29, 2022 at 10:23:55PM -0800, Nadav Amit wrote:
>>>> Using userfautlfd and looking at the kernel code, I encountered a usability
>>>> issue that complicates userspace UFFD-monitor implementation. I obviosuly
>>>> might be wrong, so I would appreciate a (polite?) feedback. I do have a
>>>> userspace workaround, but I thought it is worthy to share and to hear your
>>>> opinion, as well as feedback from other UFFD users.
>>>>
>>>> The issue I encountered regards the ordering of UFFD events tbat might not
>>>> reflect the actual order in which events took place.
>>>>
>>>> In more detail, UFFD events (e.g., unmap, fork) are not ordered against
>>>> themselves [*]. The mm-lock is dropped before notifying the userspace
>>>> UFFD-monitor, and therefore there is no guarantee as to whether the order of
>>>> the events actually reflects the order in which the events took place.
>>>> This can prevent a UFFD-monitor from using the events to track which
>>>> ranges are mapped. Specifically, UFFD_EVENT_FORK message and a
>>>> UFFD_EVENT_UNMAP message (which reflects unmap in the parent process) can
>>>> be reordered, if the events are triggered by two different threads. In
>>>> this case the UFFD-monitor cannot figure from the events whether the
>>>> child process has the unmapped memory range still mapped (because fork
>>>> happened first) or not.
>>>
>>> Yeah, it seems that something like this is possible:
>>>
>>>
>>> fork()					munmap()
>>> 	mmap_write_unlock();
>>> 						mmap_write_lock_killable();
>>> 						do_things();
>>> 						mmap_{read,write}_unlock();
>>> 						userfaultfd_unmap_complete();
>>> 	dup_userfaultfd_complete();
>>>
>>
>> I was thinking about other possible races, e.g., MADV_DONTNEED/MADV_FREE
>> racing with UFFD_EVENT_PAGEFAULT -- where we only hold the mmap_lock in
>> read mode. But not sure if they apply.
> 
> The userspace can live with these, at least for uffd missing page faults.
> If the monitor will try to resolve a page fault for a removed area, the
> errno from UFFDIO_COPY/ZERO can be used to detect such races.

I was wondering if the monitor could get confused if he just resolved a
page fault via UFFDIO_COPY/ZERO and then receives a REMOVE event.

-- 
Thanks,

David / dhildenb