[PATCH 0/4] man2: udpate mm/userfaultfd manpages to latest

[PATCH 0/4] man2: udpate mm/userfaultfd manpages to latest

[Peter Xu]

There're two features missing in current manpage, namely:

  (1) Userfaultfd Thread-ID feature
  (2) Userfaultfd write protect mode

There's also a 3rd one which was just contributed from Axel - Axel, I think it
would be great if you can add that part too, probably after the whole
hugetlbfs/shmem minor mode reaches the linux master branch.

Please review, thanks.

Peter Xu (4):
  userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs
  userfaultfd.2: Add write-protect mode
  ioctl_userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs
  ioctl_userfaultfd.2: Add write-protect mode docs

 man2/ioctl_userfaultfd.2 |  76 +++++++++++++++++++++++++++--
 man2/userfaultfd.2       | 100 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 171 insertions(+), 5 deletions(-)

-- 
2.26.2

[PATCH 1/4] userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Peter Xu]

UFFD_FEATURE_THREAD_ID is supported since Linux 4.14.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 man2/userfaultfd.2 | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index e7dc9f813..2d14effc6 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -77,6 +77,12 @@ When the last file descriptor referring to a userfaultfd object is closed,
 all memory ranges that were registered with the object are unregistered
 and unread events are flushed.
 .\"
+.PP
+Since Linux 4.14, userfaultfd page fault message can selectively embed fault
+thread ID information into the fault message.  One needs to enable this feature
+explicitly using the
+.BR UFFD_FEATURE_THREAD_ID
+feature bit when initializing the userfaultfd context, otherwise disabled.
 .SS Usage
 The userfaultfd mechanism is designed to allow a thread in a multithreaded
 program to perform user-space paging for the other threads in the process.
@@ -229,6 +235,9 @@ struct uffd_msg {
         struct {
             __u64 flags;    /* Flags describing fault */
             __u64 address;  /* Faulting address */
+            union {
+                __u32 ptid; /* Thread ID of the fault */
+            } feat;
         } pagefault;
 
         struct {            /* Since Linux 4.11 */
@@ -358,6 +367,9 @@ otherwise it is a read fault.
 .\" UFFD_PAGEFAULT_FLAG_WP is not yet supported.
 .RE
 .TP
+.I pagefault.feat.pid
+The thread ID that triggered the page fault.
+.TP
 .I fork.ufd
 The file descriptor associated with the userfault object
 created for the child created by
-- 
2.26.2

Re: [PATCH 1/4] userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Mike Rapoport]

On Wed, Mar 03, 2021 at 08:59:44PM -0500, Peter Xu wrote:
> UFFD_FEATURE_THREAD_ID is supported since Linux 4.14.
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  man2/userfaultfd.2 | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index e7dc9f813..2d14effc6 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -77,6 +77,12 @@ When the last file descriptor referring to a userfaultfd object is closed,
>  all memory ranges that were registered with the object are unregistered
>  and unread events are flushed.
>  .\"
> +.PP
> +Since Linux 4.14, userfaultfd page fault message can selectively embed fault

                                                          Maybe faulting? ^

> +thread ID information into the fault message.  One needs to enable this feature
> +explicitly using the
> +.BR UFFD_FEATURE_THREAD_ID
> +feature bit when initializing the userfaultfd context, otherwise disabled.

             "... otherwise thread ID reporting is disabled" ^

>  .SS Usage
>  The userfaultfd mechanism is designed to allow a thread in a multithreaded
>  program to perform user-space paging for the other threads in the process.
> @@ -229,6 +235,9 @@ struct uffd_msg {
>          struct {
>              __u64 flags;    /* Flags describing fault */
>              __u64 address;  /* Faulting address */
> +            union {
> +                __u32 ptid; /* Thread ID of the fault */
> +            } feat;
>          } pagefault;
> 
>          struct {            /* Since Linux 4.11 */
> @@ -358,6 +367,9 @@ otherwise it is a read fault.
>  .\" UFFD_PAGEFAULT_FLAG_WP is not yet supported.
>  .RE
>  .TP
> +.I pagefault.feat.pid
> +The thread ID that triggered the page fault.
> +.TP
>  .I fork.ufd
>  The file descriptor associated with the userfault object
>  created for the child created by
> -- 
> 2.26.2
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH 1/4] userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Peter Xu]

On Thu, Mar 04, 2021 at 08:38:09AM +0200, Mike Rapoport wrote:
> On Wed, Mar 03, 2021 at 08:59:44PM -0500, Peter Xu wrote:
> > UFFD_FEATURE_THREAD_ID is supported since Linux 4.14.
> > 
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > ---
> >  man2/userfaultfd.2 | 12 ++++++++++++
> >  1 file changed, 12 insertions(+)
> > 
> > diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> > index e7dc9f813..2d14effc6 100644
> > --- a/man2/userfaultfd.2
> > +++ b/man2/userfaultfd.2
> > @@ -77,6 +77,12 @@ When the last file descriptor referring to a userfaultfd object is closed,
> >  all memory ranges that were registered with the object are unregistered
> >  and unread events are flushed.
> >  .\"
> > +.PP
> > +Since Linux 4.14, userfaultfd page fault message can selectively embed fault
> 
>                                                           Maybe faulting? ^

Ok.

> 
> > +thread ID information into the fault message.  One needs to enable this feature
> > +explicitly using the
> > +.BR UFFD_FEATURE_THREAD_ID
> > +feature bit when initializing the userfaultfd context, otherwise disabled.
> 
>              "... otherwise thread ID reporting is disabled" ^

I changed to "... By default, thread ID reporting is diabled."

Thanks!

-- 
Peter Xu

Re: [PATCH 1/4] userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Alejandro Colomar (man-pages)]

Hello Peter,

On 3/4/21 2:59 AM, Peter Xu wrote:
> UFFD_FEATURE_THREAD_ID is supported since Linux 4.14.
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>   man2/userfaultfd.2 | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
> 
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index e7dc9f813..2d14effc6 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -77,6 +77,12 @@ When the last file descriptor referring to a userfaultfd object is closed,
>   all memory ranges that were registered with the object are unregistered
>   and unread events are flushed.
>   .\"
> +.PP
> +Since Linux 4.14, userfaultfd page fault message can selectively embed fault
> +thread ID information into the fault message.  One needs to enable this feature
> +explicitly using the
> +.BR UFFD_FEATURE_THREAD_ID

This should use [.B] and not [.BR].
.BR is for alternate Bold and Roman.
.B is for bold.

(There are more appearances of this in the other patches.)

Thanks,

Alex

-- 
Alejandro Colomar
Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/

Re: [PATCH 1/4] userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Peter Xu]

On Thu, Mar 04, 2021 at 10:22:18AM +0100, Alejandro Colomar (man-pages) wrote:
> Hello Peter,

Hi, Alex,

> 
> On 3/4/21 2:59 AM, Peter Xu wrote:
> > UFFD_FEATURE_THREAD_ID is supported since Linux 4.14.
> > 
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > ---
> >   man2/userfaultfd.2 | 12 ++++++++++++
> >   1 file changed, 12 insertions(+)
> > 
> > diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> > index e7dc9f813..2d14effc6 100644
> > --- a/man2/userfaultfd.2
> > +++ b/man2/userfaultfd.2
> > @@ -77,6 +77,12 @@ When the last file descriptor referring to a userfaultfd object is closed,
> >   all memory ranges that were registered with the object are unregistered
> >   and unread events are flushed.
> >   .\"
> > +.PP
> > +Since Linux 4.14, userfaultfd page fault message can selectively embed fault
> > +thread ID information into the fault message.  One needs to enable this feature
> > +explicitly using the
> > +.BR UFFD_FEATURE_THREAD_ID
> 
> This should use [.B] and not [.BR].
> .BR is for alternate Bold and Roman.
> .B is for bold.
> 
> (There are more appearances of this in the other patches.)

Yeah I got a bit confused when differenciating those two, since I also see
similar usage, e.g.:

.BR O_CLOEXEC

I'll fix all of them appeared in current patchset.  Let me know if you also
want me to fix all the existing uses of ".BR" too where ".B" would suffice.
Otherwise I won't touch them since I can't say they're wrong either (I think
most of them should generate the same output with either ".BR" or ".B" if
there's only one word?).

Thanks,

-- 
Peter Xu

Re: [PATCH 1/4] userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Alejandro Colomar (man-pages)]

Hi Peter,

On 3/4/21 4:50 PM, Peter Xu wrote:
> On Thu, Mar 04, 2021 at 10:22:18AM +0100, Alejandro Colomar (man-pages) wrote:
>>> +.BR UFFD_FEATURE_THREAD_ID
>>
>> This should use [.B] and not [.BR].
>> .BR is for alternate Bold and Roman.
>> .B is for bold.
>>
>> (There are more appearances of this in the other patches.)
> 
> Yeah I got a bit confused when differenciating those two, since I also see
> similar usage, e.g.:
> 
> .BR O_CLOEXEC

Yes, these are minor imperfections that got into the manual pages, and 
we don't remove them due to the churn that it would create (and 
possibility of introducing other bugs while doing such a big scripted 
change that couldn't be easily reviewed (thousands of lines)).  So as we 
still have those lines, they tend to confuse.

> 
> I'll fix all of them appeared in current patchset.  Let me know if you also
> want me to fix all the existing uses of ".BR" too where ".B" would suffice.
> Otherwise I won't touch them since I can't say they're wrong either (I think
> most of them should generate the same output with either ".BR" or ".B" if
> there's only one word?).

Our current non-written guidelines are:
We are fixing the existing ones as we modify code near them,
but leave untouched code that is far from what we are changing, even on 
the same page.

Thanks,

Alex

-- 
Alejandro Colomar
Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/

[PATCH 2/4] userfaultfd.2: Add write-protect mode

[Peter Xu]

Write-protect mode is supported starting from Linux 5.7.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 man2/userfaultfd.2 | 88 ++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 86 insertions(+), 2 deletions(-)

diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index 2d14effc6..8e1602d62 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -78,6 +78,28 @@ all memory ranges that were registered with the object are unregistered
 and unread events are flushed.
 .\"
 .PP
+Currently, userfaultfd supports two modes of registration:
+.TP
+.BR UFFDIO_REGISTER_MODE_MISSING
+When registered with
+.BR UFFDIO_REGISTER_MODE_MISSING
+mode, the userspace will receive a page fault message when a missing page is
+accessed.  The faulted thread will be stopped from execution until the page
+fault is resolved from the userspace by either an
+.BR UFFDIO_COPY
+or an
+.BR UFFDIO_ZEROPAGE
+ioctl.
+.TP
+.BR UFFDIO_REGISTER_MODE_WP
+When registered with
+.BR UFFDIO_REGISTER_MODE_WP
+mode, the userspace will receive a page fault message when a write-protected
+page is written.  The faulted thread will be stopped from execution until the
+userspace un-write-protect the page using an
+.BR UFFDIO_WRITEPROTECT
+ioctl.
+.PP
 Since Linux 4.14, userfaultfd page fault message can selectively embed fault
 thread ID information into the fault message.  One needs to enable this feature
 explicitly using the
@@ -143,6 +165,16 @@ single threaded non-cooperative userfaultfd manager implementations.
 .\" and limitations remaining in 4.11
 .\" Maybe it's worth adding a dedicated sub-section...
 .\"
+.PP
+Starting from Linux 5.7, userfaultfd is able to do synchronous page dirty
+tracking using the new write-protection register mode.  One should check
+against the feature bit
+.B UFFD_FEATURE_PAGEFAULT_FLAG_WP
+before using this feature.  Similar to the original userfaultfd missing mode,
+the write-protect mode will generate an userfaultfd message when the protected
+page is written.  The user needs to resolve the page fault by unprotecting the
+faulted page and kick the faulted thread to continue.  For more information,
+please read the "Userfaultfd write-protect mode" section below.
 .SS Userfaultfd operation
 After the userfaultfd object is created with
 .BR userfaultfd (),
@@ -218,6 +250,54 @@ userfaultfd can be used only with anonymous private memory mappings.
 Since Linux 4.11,
 userfaultfd can be also used with hugetlbfs and shared memory mappings.
 .\"
+.SS Userfaultfd write-protect mode
+Since Linux 5.7, userfaultfd started to support write-protect mode.  The user
+needs to first check availability of this feature using
+.BR UFFDIO_API
+ioctl against the feature bit
+.BR UFFD_FEATURE_PAGEFAULT_FLAG_WP .
+.PP
+To register with userfaultfd write-protect mode, the user needs to send the
+.BR UFFDIO_REGISTER
+ioctl with mode
+.BR UFFDIO_REGISTER_MODE_WP
+set.  Note that it's legal to monitor the same memory range with multiple
+modes.  For example, the user can do
+.BR UFFDIO_REGISTER
+with the mode set to
+.BR UFFDIO_REGISTER_MODE_MISSING\ |\ UFFDIO_REGISTER_MODE_WP.
+When there is only
+.BR UFFDIO_REGISTER_MODE_WP
+registered, the userspace will
+.I not
+receive any message when a missing page is written.  Instead, the userspace
+will only receive a write-protect page fault message when an existing but
+write-protected page got written.
+.PP
+After the
+.BR UFFDIO_REGISTER
+ioctl completed with
+.BR UFFDIO_REGISTER_MODE_WP
+mode set, one can write-protect any existing memory within the range using the
+ioctl
+.BR UFFDIO_WRITEPROTECT
+where
+.I uffdio_writeprotect.mode
+should be set to
+.BR UFFDIO_WRITEPROTECT_MODE_WP .
+.PP
+When a write-protect event happens, the userspace will receive a page fault
+message whose
+.I uffd_msg.pagefault.flags
+will be with
+.BR UFFD_PAGEFAULT_FLAG_WP
+flag set.  Note: since only writes can trigger such kind of fault,
+write-protect messages will always be with
+.BR UFFD_PAGEFAULT_FLAG_WRITE
+bit set too along with
+.BR UFFD_PAGEFAULT_FLAG_WP .
+.PP
+Currently, write-protect mode only supports private anonymous memory.
 .SS Reading from the userfaultfd structure
 Each
 .BR read (2)
@@ -363,8 +443,12 @@ flag (see
 .BR ioctl_userfaultfd (2))
 and this flag is set, this a write fault;
 otherwise it is a read fault.
-.\"
-.\" UFFD_PAGEFAULT_FLAG_WP is not yet supported.
+.TP
+.B UFFD_PAGEFAULT_FLAG_WP
+If the address is in a range that was registered with the
+.B UFFDIO_REGISTER_MODE_WP
+flag, when this bit is set it means it's a write-protect fault.  Otherwise it's
+a page missing fault.
 .RE
 .TP
 .I pagefault.feat.pid
-- 
2.26.2

Re: [PATCH 2/4] userfaultfd.2: Add write-protect mode

[Mike Rapoport]

On Wed, Mar 03, 2021 at 08:59:45PM -0500, Peter Xu wrote:
> Write-protect mode is supported starting from Linux 5.7.
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  man2/userfaultfd.2 | 88 ++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 86 insertions(+), 2 deletions(-)
> 
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index 2d14effc6..8e1602d62 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -78,6 +78,28 @@ all memory ranges that were registered with the object are unregistered
>  and unread events are flushed.
>  .\"
>  .PP
> +Currently, userfaultfd supports two modes of registration:
> +.TP
> +.BR UFFDIO_REGISTER_MODE_MISSING
> +When registered with
> +.BR UFFDIO_REGISTER_MODE_MISSING
> +mode, the userspace will receive a page fault message when a missing page is
> +accessed.  The faulted thread will be stopped from execution until the page
> +fault is resolved from the userspace by either an
> +.BR UFFDIO_COPY
> +or an
> +.BR UFFDIO_ZEROPAGE
> +ioctl.
> +.TP
> +.BR UFFDIO_REGISTER_MODE_WP
> +When registered with
> +.BR UFFDIO_REGISTER_MODE_WP
> +mode, the userspace will receive a page fault message when a write-protected
> +page is written.  The faulted thread will be stopped from execution until the
> +userspace un-write-protect the page using an
> +.BR UFFDIO_WRITEPROTECT
> +ioctl.
> +.PP

I'd add a sentence about combining the modes together. Something like

"Both modes can be enabled together for the same memory range"

>  Since Linux 4.14, userfaultfd page fault message can selectively embed fault
>  thread ID information into the fault message.  One needs to enable this feature
>  explicitly using the
> @@ -143,6 +165,16 @@ single threaded non-cooperative userfaultfd manager implementations.
>  .\" and limitations remaining in 4.11
>  .\" Maybe it's worth adding a dedicated sub-section...
>  .\"
> +.PP
> +Starting from Linux 5.7, userfaultfd is able to do synchronous page dirty
> +tracking using the new write-protection register mode.  One should check
> +against the feature bit
> +.B UFFD_FEATURE_PAGEFAULT_FLAG_WP
> +before using this feature.  Similar to the original userfaultfd missing mode,
> +the write-protect mode will generate an userfaultfd message when the protected
> +page is written.  The user needs to resolve the page fault by unprotecting the
> +faulted page and kick the faulted thread to continue.  For more information,
> +please read the "Userfaultfd write-protect mode" section below.
>  .SS Userfaultfd operation
>  After the userfaultfd object is created with
>  .BR userfaultfd (),
> @@ -218,6 +250,54 @@ userfaultfd can be used only with anonymous private memory mappings.
>  Since Linux 4.11,
>  userfaultfd can be also used with hugetlbfs and shared memory mappings.
>  .\"
> +.SS Userfaultfd write-protect mode
> +Since Linux 5.7, userfaultfd started to support write-protect mode.  The user

Maybe s/started to support/supports/

> +needs to first check availability of this feature using
> +.BR UFFDIO_API
> +ioctl against the feature bit
> +.BR UFFD_FEATURE_PAGEFAULT_FLAG_WP .
> +.PP
> +To register with userfaultfd write-protect mode, the user needs to send the
> +.BR UFFDIO_REGISTER
> +ioctl with mode
> +.BR UFFDIO_REGISTER_MODE_WP
> +set.  Note that it's legal to monitor the same memory range with multiple
> +modes.  For example, the user can do
> +.BR UFFDIO_REGISTER
> +with the mode set to
> +.BR UFFDIO_REGISTER_MODE_MISSING\ |\ UFFDIO_REGISTER_MODE_WP.
> +When there is only
> +.BR UFFDIO_REGISTER_MODE_WP
> +registered, the userspace will
> +.I not
> +receive any message when a missing page is written.  Instead, the userspace
> +will only receive a write-protect page fault message when an existing but
> +write-protected page got written.
> +.PP
> +After the
> +.BR UFFDIO_REGISTER
> +ioctl completed with
> +.BR UFFDIO_REGISTER_MODE_WP
> +mode set, one can write-protect any existing memory within the range using the
> +ioctl
> +.BR UFFDIO_WRITEPROTECT
> +where
> +.I uffdio_writeprotect.mode
> +should be set to
> +.BR UFFDIO_WRITEPROTECT_MODE_WP .
> +.PP
> +When a write-protect event happens, the userspace will receive a page fault
> +message whose
> +.I uffd_msg.pagefault.flags
> +will be with
> +.BR UFFD_PAGEFAULT_FLAG_WP
> +flag set.  Note: since only writes can trigger such kind of fault,
> +write-protect messages will always be with
> +.BR UFFD_PAGEFAULT_FLAG_WRITE
> +bit set too along with
> +.BR UFFD_PAGEFAULT_FLAG_WP .
> +.PP
> +Currently, write-protect mode only supports private anonymous memory.
>  .SS Reading from the userfaultfd structure
>  Each
>  .BR read (2)
> @@ -363,8 +443,12 @@ flag (see
>  .BR ioctl_userfaultfd (2))
>  and this flag is set, this a write fault;
>  otherwise it is a read fault.
> -.\"
> -.\" UFFD_PAGEFAULT_FLAG_WP is not yet supported.
> +.TP
> +.B UFFD_PAGEFAULT_FLAG_WP
> +If the address is in a range that was registered with the
> +.B UFFDIO_REGISTER_MODE_WP
> +flag, when this bit is set it means it's a write-protect fault.  Otherwise it's
> +a page missing fault.
>  .RE
>  .TP
>  .I pagefault.feat.pid
> -- 
> 2.26.2
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH 2/4] userfaultfd.2: Add write-protect mode

[Peter Xu]

On Thu, Mar 04, 2021 at 08:46:59AM +0200, Mike Rapoport wrote:
> On Wed, Mar 03, 2021 at 08:59:45PM -0500, Peter Xu wrote:
> > Write-protect mode is supported starting from Linux 5.7.
> > 
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > ---
> >  man2/userfaultfd.2 | 88 ++++++++++++++++++++++++++++++++++++++++++++--
> >  1 file changed, 86 insertions(+), 2 deletions(-)
> > 
> > diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> > index 2d14effc6..8e1602d62 100644
> > --- a/man2/userfaultfd.2
> > +++ b/man2/userfaultfd.2
> > @@ -78,6 +78,28 @@ all memory ranges that were registered with the object are unregistered
> >  and unread events are flushed.
> >  .\"
> >  .PP
> > +Currently, userfaultfd supports two modes of registration:
> > +.TP
> > +.BR UFFDIO_REGISTER_MODE_MISSING
> > +When registered with
> > +.BR UFFDIO_REGISTER_MODE_MISSING
> > +mode, the userspace will receive a page fault message when a missing page is
> > +accessed.  The faulted thread will be stopped from execution until the page
> > +fault is resolved from the userspace by either an
> > +.BR UFFDIO_COPY
> > +or an
> > +.BR UFFDIO_ZEROPAGE
> > +ioctl.
> > +.TP
> > +.BR UFFDIO_REGISTER_MODE_WP
> > +When registered with
> > +.BR UFFDIO_REGISTER_MODE_WP
> > +mode, the userspace will receive a page fault message when a write-protected
> > +page is written.  The faulted thread will be stopped from execution until the
> > +userspace un-write-protect the page using an
> > +.BR UFFDIO_WRITEPROTECT
> > +ioctl.
> > +.PP
> 
> I'd add a sentence about combining the modes together. Something like
> 
> "Both modes can be enabled together for the same memory range"

I mentioned it below [1].  However I agree it's indeed making more sense to
mention it when listing the modes, especially knowing that the 3rd minor mode
is coming.  I think I'll keep both, assuming a bit more verbose is still
acceptable in man pages, but changed to:

"Multiple modes can be enabled at the same time for the same memory range."

> 
> >  Since Linux 4.14, userfaultfd page fault message can selectively embed fault
> >  thread ID information into the fault message.  One needs to enable this feature
> >  explicitly using the
> > @@ -143,6 +165,16 @@ single threaded non-cooperative userfaultfd manager implementations.
> >  .\" and limitations remaining in 4.11
> >  .\" Maybe it's worth adding a dedicated sub-section...
> >  .\"
> > +.PP
> > +Starting from Linux 5.7, userfaultfd is able to do synchronous page dirty
> > +tracking using the new write-protection register mode.  One should check
> > +against the feature bit
> > +.B UFFD_FEATURE_PAGEFAULT_FLAG_WP
> > +before using this feature.  Similar to the original userfaultfd missing mode,
> > +the write-protect mode will generate an userfaultfd message when the protected
> > +page is written.  The user needs to resolve the page fault by unprotecting the
> > +faulted page and kick the faulted thread to continue.  For more information,
> > +please read the "Userfaultfd write-protect mode" section below.
> >  .SS Userfaultfd operation
> >  After the userfaultfd object is created with
> >  .BR userfaultfd (),
> > @@ -218,6 +250,54 @@ userfaultfd can be used only with anonymous private memory mappings.
> >  Since Linux 4.11,
> >  userfaultfd can be also used with hugetlbfs and shared memory mappings.
> >  .\"
> > +.SS Userfaultfd write-protect mode
> > +Since Linux 5.7, userfaultfd started to support write-protect mode.  The user
> 
> Maybe s/started to support/supports/

Sure.

> 
> > +needs to first check availability of this feature using
> > +.BR UFFDIO_API
> > +ioctl against the feature bit
> > +.BR UFFD_FEATURE_PAGEFAULT_FLAG_WP .
> > +.PP
> > +To register with userfaultfd write-protect mode, the user needs to send the
> > +.BR UFFDIO_REGISTER
> > +ioctl with mode
> > +.BR UFFDIO_REGISTER_MODE_WP
> > +set.  Note that it's legal to monitor the same memory range with multiple
> > +modes.  For example, the user can do
> > +.BR UFFDIO_REGISTER
> > +with the mode set to
> > +.BR UFFDIO_REGISTER_MODE_MISSING\ |\ UFFDIO_REGISTER_MODE_WP.

[1]

Thanks,

-- 
Peter Xu

[PATCH 3/4] ioctl_userfaultfd.2: Add UFFD_FEATURE_THREAD_ID docs

[Peter Xu]

UFFD_FEATURE_THREAD_ID is supported in Linux 4.14.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 man2/ioctl_userfaultfd.2 | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 47ae5f473..1965d1932 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -208,6 +208,10 @@ signal will be sent to the faulting process.
 Applications using this
 feature will not require the use of a userfaultfd monitor for processing
 memory accesses to the regions registered with userfaultfd.
+.BR UFFD_FEATURE_THREAD_ID " (since Linux 4.14)"
+If this feature bit is set,
+.I uffd_msg.pagefault.feat.ptid
+Will be set
 .PP
 The returned
 .I ioctls
-- 
2.26.2

[PATCH 4/4] ioctl_userfaultfd.2: Add write-protect mode docs

[Peter Xu]

Userfaultfd write-protect mode is supported starting from Linux 5.7.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 man2/ioctl_userfaultfd.2 | 74 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 70 insertions(+), 4 deletions(-)

diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 1965d1932..3feb888a8 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -208,10 +208,11 @@ signal will be sent to the faulting process.
 Applications using this
 feature will not require the use of a userfaultfd monitor for processing
 memory accesses to the regions registered with userfaultfd.
+.TP
 .BR UFFD_FEATURE_THREAD_ID " (since Linux 4.14)"
 If this feature bit is set,
 .I uffd_msg.pagefault.feat.ptid
-Will be set
+will be set to the faulted thread ID for each page fault message.
 .PP
 The returned
 .I ioctls
@@ -233,6 +234,11 @@ operation is supported.
 The
 .B UFFDIO_UNREGISTER
 operation is supported.
+.TP
+.B 1 << _UFFDIO_WRITEPROTECT
+The
+.B UFFDIO_WRITEPROTECT
+operation is supported.
 .PP
 This
 .BR ioctl (2)
@@ -321,9 +327,6 @@ Track page faults on missing pages.
 .B UFFDIO_REGISTER_MODE_WP
 Track page faults on write-protected pages.
 .PP
-Currently, the only supported mode is
-.BR UFFDIO_REGISTER_MODE_MISSING .
-.PP
 If the operation is successful, the kernel modifies the
 .I ioctls
 bit-mask field to indicate which
@@ -653,6 +656,69 @@ field of the
 structure was not a multiple of the system page size; or
 .I len
 was zero; or the specified range was otherwise invalid.
+.SS UFFDIO_WRITEPROTECT
+(Since Linux 5.7) Do write-protect or write-unprotect for an userfaultfd
+registered memory range with mode
+.BR UFFDIO_REGISTER_MODE_WP .
+.PP
+The
+.I argp
+argument is a pointer to a
+.I uffdio_range
+structure as shown below:
+.PP
+.in +4n
+.EX
+struct uffdio_writeprotect {
+    struct uffdio_range range;  /* Range to change write permission */
+    __u64 mode;                 /* Mode to change write permission */
+};
+.EE
+.in
+There're two modes that are supported in this structure:
+.TP
+.B UFFDIO_WRITEPROTECT_MODE_WP
+When this mode bit is set, the ioctl will be a write-protect operation upon the
+memory range specified by
+.IR range .
+Otherwise it'll be a write-unprotect operation upon the specified range.
+.TP
+.B UFFDIO_WRITEPROTECT_MODE_DONTWAKE
+Do not wake up the thread that waits for page-fault resolution after the
+operation.  This could only be specified if
+.B UFFDIO_WRITEPROTECT_MODE_WP
+is not specified (in a resolving stage, not protecting stage).
+.PP
+This
+.BR ioctl (2)
+operation returns 0 on success.
+On error, \-1 is returned and
+.I errno
+is set to indicate the error.
+Possible errors include:
+.TP
+.B EINVAL
+The
+.I start
+or the
+.I len
+field of the
+.I ufdio_range
+structure was not a multiple of the system page size; or
+.I len
+was zero; or the specified range was otherwise invalid.
+.TP
+.B EAGAIN
+The process was interrupted and need to retry.
+.TP
+.B ENOENT
+The range specified in
+.I range
+is not valid.  E.g., the virtual address does not exist, or not registered with
+userfaultfd write-protect mode.
+.TP
+.B EFAULT
+Encountered a generic fault during processing.
 .SH RETURN VALUE
 See descriptions of the individual operations, above.
 .SH ERRORS
-- 
2.26.2

Re: [PATCH 4/4] ioctl_userfaultfd.2: Add write-protect mode docs

[Peter Xu]

On Wed, Mar 03, 2021 at 08:59:47PM -0500, Peter Xu wrote:
> Userfaultfd write-protect mode is supported starting from Linux 5.7.
> 
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>  man2/ioctl_userfaultfd.2 | 74 +++++++++++++++++++++++++++++++++++++---
>  1 file changed, 70 insertions(+), 4 deletions(-)
> 
> diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> index 1965d1932..3feb888a8 100644
> --- a/man2/ioctl_userfaultfd.2
> +++ b/man2/ioctl_userfaultfd.2
> @@ -208,10 +208,11 @@ signal will be sent to the faulting process.
>  Applications using this
>  feature will not require the use of a userfaultfd monitor for processing
>  memory accesses to the regions registered with userfaultfd.
> +.TP
>  .BR UFFD_FEATURE_THREAD_ID " (since Linux 4.14)"
>  If this feature bit is set,
>  .I uffd_msg.pagefault.feat.ptid
> -Will be set
> +will be set to the faulted thread ID for each page fault message.

These two changes should belong to the previous patch - will get them fixed
too.

-- 
Peter Xu