[PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[Sasha Levin]

From: Matthew Bobrowski <mbobrowski@mbobrowski.org>

[ Upstream commit 2d10b23082a7eb8be508b3789f2e7250a88a5ddb ]

Modify fanotify_should_send_event() so that it now returns a mask for
an event that contains ONLY flags for the event types that have been
specifically requested by the user. Flags that may have been included
within the event mask, but have not been explicitly requested by the
user will not be present in the returned value.

As an example, given the situation where a user requests events of type
FAN_OPEN. Traditionally, the event mask returned within an event that
occurred on a filesystem object that has been marked for monitoring and is
opened, will only ever have the FAN_OPEN bit set. With the introduction of
the new flags like FAN_OPEN_EXEC, and perhaps any other future event
flags, there is a possibility of the returned event mask containing more
than a single bit set, despite having only requested the single event type.
Prior to these modifications performed to fanotify_should_send_event(), a
user would have received a bundled event mask containing flags FAN_OPEN
and FAN_OPEN_EXEC in the instance that a file was opened for execution via
execve(), for example. This means that a user would receive event types
in the returned event mask that have not been requested. This runs the
possibility of breaking existing systems and causing other unforeseen
issues.

To mitigate this possibility, fanotify_should_send_event() has been
modified to return the event mask containing ONLY event types explicitly
requested by the user. This means that we will NOT report events that the
user did no set a mask for, and we will NOT report events that the user
has set an ignore mask for.

The function name fanotify_should_send_event() has also been updated so
that it's more relevant to what it has been designed to do.

Signed-off-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 fs/notify/fanotify/fanotify.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index e08a6647267b..f4f8359bc597 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -89,7 +89,13 @@ static int fanotify_get_response(struct fsnotify_group *group,
 	return ret;
 }
 
-static bool fanotify_should_send_event(struct fsnotify_iter_info *iter_info,
+/*
+ * This function returns a mask for an event that only contains the flags
+ * that have been specifically requested by the user. Flags that may have
+ * been included within the event mask, but have not been explicitly
+ * requested by the user, will not be present in the returned mask.
+ */
+static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 				       u32 event_mask, const void *data,
 				       int data_type)
 {
@@ -101,14 +107,14 @@ static bool fanotify_should_send_event(struct fsnotify_iter_info *iter_info,
 	pr_debug("%s: report_mask=%x mask=%x data=%p data_type=%d\n",
 		 __func__, iter_info->report_mask, event_mask, data, data_type);
 
-	/* if we don't have enough info to send an event to userspace say no */
+	/* If we don't have enough info to send an event to userspace say no */
 	if (data_type != FSNOTIFY_EVENT_PATH)
-		return false;
+		return 0;
 
-	/* sorry, fanotify only gives a damn about files and dirs */
+	/* Sorry, fanotify only gives a damn about files and dirs */
 	if (!d_is_reg(path->dentry) &&
 	    !d_can_lookup(path->dentry))
-		return false;
+		return 0;
 
 	fsnotify_foreach_obj_type(type) {
 		if (!fsnotify_iter_should_report_type(iter_info, type))
@@ -129,13 +135,10 @@ static bool fanotify_should_send_event(struct fsnotify_iter_info *iter_info,
 
 	if (d_is_dir(path->dentry) &&
 	    !(marks_mask & FS_ISDIR & ~marks_ignored_mask))
-		return false;
-
-	if (event_mask & FANOTIFY_OUTGOING_EVENTS &
-	    marks_mask & ~marks_ignored_mask)
-		return true;
+		return 0;
 
-	return false;
+	return event_mask & FANOTIFY_OUTGOING_EVENTS & marks_mask &
+		~marks_ignored_mask;
 }
 
 struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
@@ -210,7 +213,8 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 
 	BUILD_BUG_ON(HWEIGHT32(ALL_FANOTIFY_EVENT_BITS) != 10);
 
-	if (!fanotify_should_send_event(iter_info, mask, data, data_type))
+	mask = fanotify_group_event_mask(iter_info, mask, data, data_type);
+	if (!mask)
 		return 0;
 
 	pr_debug("%s: group=%p inode=%p mask=%x\n", __func__, group, inode,
-- 
2.19.1

Re: [PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[Amir Goldstein]

On Tue, Jan 8, 2019 at 10:11 PM Sasha Levin <sashal@kernel.org> wrote:
>
> From: Matthew Bobrowski <mbobrowski@mbobrowski.org>
>
> [ Upstream commit 2d10b23082a7eb8be508b3789f2e7250a88a5ddb ]
>
> Modify fanotify_should_send_event() so that it now returns a mask for
> an event that contains ONLY flags for the event types that have been
> specifically requested by the user. Flags that may have been included
> within the event mask, but have not been explicitly requested by the
> user will not be present in the returned value.
>
> As an example, given the situation where a user requests events of type
> FAN_OPEN. Traditionally, the event mask returned within an event that
> occurred on a filesystem object that has been marked for monitoring and is
> opened, will only ever have the FAN_OPEN bit set. With the introduction of
> the new flags like FAN_OPEN_EXEC, and perhaps any other future event
> flags, there is a possibility of the returned event mask containing more
> than a single bit set, despite having only requested the single event type.
> Prior to these modifications performed to fanotify_should_send_event(), a
> user would have received a bundled event mask containing flags FAN_OPEN
> and FAN_OPEN_EXEC in the instance that a file was opened for execution via
> execve(), for example. This means that a user would receive event types
> in the returned event mask that have not been requested. This runs the
> possibility of breaking existing systems and causing other unforeseen
> issues.
>
> To mitigate this possibility, fanotify_should_send_event() has been
> modified to return the event mask containing ONLY event types explicitly
> requested by the user. This means that we will NOT report events that the
> user did no set a mask for, and we will NOT report events that the user
> has set an ignore mask for.
>
> The function name fanotify_should_send_event() has also been updated so
> that it's more relevant to what it has been designed to do.
>
> Signed-off-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> Signed-off-by: Jan Kara <jack@suse.cz>
> Signed-off-by: Sasha Levin <sashal@kernel.org>
> ---

Sasha,

I have no objection to applying this patch to 4.20, but FYI, it does
not fix anything.
Before introducing FAN_OPEN_EXEC in 5.0-rc1, this patch has no visible effect.

I don't mind if you apply it. It will make stable code closer to
mainline, which is
always a good thing IMO. And FWIW, I think that patch is quite trivial
and low risk.

Thanks,
Amir.

Re: [PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[Amir Goldstein]

On Tue, Jan 8, 2019 at 10:11 PM Sasha Levin <sashal@kernel.org> wrote:
>
> From: Matthew Bobrowski <mbobrowski@mbobrowski.org>
>
> [ Upstream commit 2d10b23082a7eb8be508b3789f2e7250a88a5ddb ]
>
> Modify fanotify_should_send_event() so that it now returns a mask for
> an event that contains ONLY flags for the event types that have been
> specifically requested by the user. Flags that may have been included
> within the event mask, but have not been explicitly requested by the
> user will not be present in the returned value.
>
> As an example, given the situation where a user requests events of type
> FAN_OPEN. Traditionally, the event mask returned within an event that
> occurred on a filesystem object that has been marked for monitoring and is
> opened, will only ever have the FAN_OPEN bit set. With the introduction of
> the new flags like FAN_OPEN_EXEC, and perhaps any other future event
> flags, there is a possibility of the returned event mask containing more
> than a single bit set, despite having only requested the single event type.
> Prior to these modifications performed to fanotify_should_send_event(), a
> user would have received a bundled event mask containing flags FAN_OPEN
> and FAN_OPEN_EXEC in the instance that a file was opened for execution via
> execve(), for example. This means that a user would receive event types
> in the returned event mask that have not been requested. This runs the
> possibility of breaking existing systems and causing other unforeseen
> issues.
>
> To mitigate this possibility, fanotify_should_send_event() has been
> modified to return the event mask containing ONLY event types explicitly
> requested by the user. This means that we will NOT report events that the
> user did no set a mask for, and we will NOT report events that the user
> has set an ignore mask for.
>
> The function name fanotify_should_send_event() has also been updated so
> that it's more relevant to what it has been designed to do.
>
> Signed-off-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> Signed-off-by: Jan Kara <jack@suse.cz>
> Signed-off-by: Sasha Levin <sashal@kernel.org>
> ---

Sasha,

I have no objection to applying this patch to 4.20, but FYI, it does
not fix anything.
Before introducing FAN_OPEN_EXEC in 5.0-rc1, this patch has no visible effect.

I don't mind if you apply it. It will make stable code closer to
mainline, which is
always a good thing IMO. And FWIW, I think that patch is quite trivial
and low risk.

Thanks,
Amir.

Re: [PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[Jan Kara]

On Wed 09-01-19 08:50:33, Amir Goldstein wrote:
> On Tue, Jan 8, 2019 at 10:11 PM Sasha Levin <sashal@kernel.org> wrote:
> >
> > From: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> >
> > [ Upstream commit 2d10b23082a7eb8be508b3789f2e7250a88a5ddb ]
> >
> > Modify fanotify_should_send_event() so that it now returns a mask for
> > an event that contains ONLY flags for the event types that have been
> > specifically requested by the user. Flags that may have been included
> > within the event mask, but have not been explicitly requested by the
> > user will not be present in the returned value.
> >
> > As an example, given the situation where a user requests events of type
> > FAN_OPEN. Traditionally, the event mask returned within an event that
> > occurred on a filesystem object that has been marked for monitoring and is
> > opened, will only ever have the FAN_OPEN bit set. With the introduction of
> > the new flags like FAN_OPEN_EXEC, and perhaps any other future event
> > flags, there is a possibility of the returned event mask containing more
> > than a single bit set, despite having only requested the single event type.
> > Prior to these modifications performed to fanotify_should_send_event(), a
> > user would have received a bundled event mask containing flags FAN_OPEN
> > and FAN_OPEN_EXEC in the instance that a file was opened for execution via
> > execve(), for example. This means that a user would receive event types
> > in the returned event mask that have not been requested. This runs the
> > possibility of breaking existing systems and causing other unforeseen
> > issues.
> >
> > To mitigate this possibility, fanotify_should_send_event() has been
> > modified to return the event mask containing ONLY event types explicitly
> > requested by the user. This means that we will NOT report events that the
> > user did no set a mask for, and we will NOT report events that the user
> > has set an ignore mask for.
> >
> > The function name fanotify_should_send_event() has also been updated so
> > that it's more relevant to what it has been designed to do.
> >
> > Signed-off-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> > Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > Signed-off-by: Sasha Levin <sashal@kernel.org>
> > ---
> 
> I have no objection to applying this patch to 4.20, but FYI, it does not
> fix anything.  Before introducing FAN_OPEN_EXEC in 5.0-rc1, this patch
> has no visible effect.

Yes, the patch is just a code refactoring useful for the FAN_OPEN_EXEC
feature.

> I don't mind if you apply it. It will make stable code closer to
> mainline, which is always a good thing IMO. And FWIW, I think that patch
> is quite trivial and low risk.

I don't think applying code refactoring to stable is a good idea. Every
change has a risk of regression and this particular one brings users no
benefit. So I'd prefer to drop this patch from stable queue.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[Jan Kara]

On Wed 09-01-19 08:50:33, Amir Goldstein wrote:
> On Tue, Jan 8, 2019 at 10:11 PM Sasha Levin <sashal@kernel.org> wrote:
> >
> > From: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> >
> > [ Upstream commit 2d10b23082a7eb8be508b3789f2e7250a88a5ddb ]
> >
> > Modify fanotify_should_send_event() so that it now returns a mask for
> > an event that contains ONLY flags for the event types that have been
> > specifically requested by the user. Flags that may have been included
> > within the event mask, but have not been explicitly requested by the
> > user will not be present in the returned value.
> >
> > As an example, given the situation where a user requests events of type
> > FAN_OPEN. Traditionally, the event mask returned within an event that
> > occurred on a filesystem object that has been marked for monitoring and is
> > opened, will only ever have the FAN_OPEN bit set. With the introduction of
> > the new flags like FAN_OPEN_EXEC, and perhaps any other future event
> > flags, there is a possibility of the returned event mask containing more
> > than a single bit set, despite having only requested the single event type.
> > Prior to these modifications performed to fanotify_should_send_event(), a
> > user would have received a bundled event mask containing flags FAN_OPEN
> > and FAN_OPEN_EXEC in the instance that a file was opened for execution via
> > execve(), for example. This means that a user would receive event types
> > in the returned event mask that have not been requested. This runs the
> > possibility of breaking existing systems and causing other unforeseen
> > issues.
> >
> > To mitigate this possibility, fanotify_should_send_event() has been
> > modified to return the event mask containing ONLY event types explicitly
> > requested by the user. This means that we will NOT report events that the
> > user did no set a mask for, and we will NOT report events that the user
> > has set an ignore mask for.
> >
> > The function name fanotify_should_send_event() has also been updated so
> > that it's more relevant to what it has been designed to do.
> >
> > Signed-off-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> > Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > Signed-off-by: Sasha Levin <sashal@kernel.org>
> > ---
> 
> I have no objection to applying this patch to 4.20, but FYI, it does not
> fix anything.  Before introducing FAN_OPEN_EXEC in 5.0-rc1, this patch
> has no visible effect.

Yes, the patch is just a code refactoring useful for the FAN_OPEN_EXEC
feature.

> I don't mind if you apply it. It will make stable code closer to
> mainline, which is always a good thing IMO. And FWIW, I think that patch
> is quite trivial and low risk.

I don't think applying code refactoring to stable is a good idea. Every
change has a risk of regression and this particular one brings users no
benefit. So I'd prefer to drop this patch from stable queue.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH AUTOSEL 4.20 016/117] fanotify: return only user requested event types in event mask

[Sasha Levin]

On Wed, Jan 09, 2019 at 12:32:42PM +0100, Jan Kara wrote:
>On Wed 09-01-19 08:50:33, Amir Goldstein wrote:
>> On Tue, Jan 8, 2019 at 10:11 PM Sasha Levin <sashal@kernel.org> wrote:
>> >
>> > From: Matthew Bobrowski <mbobrowski@mbobrowski.org>
>> >
>> > [ Upstream commit 2d10b23082a7eb8be508b3789f2e7250a88a5ddb ]
>> >
>> > Modify fanotify_should_send_event() so that it now returns a mask for
>> > an event that contains ONLY flags for the event types that have been
>> > specifically requested by the user. Flags that may have been included
>> > within the event mask, but have not been explicitly requested by the
>> > user will not be present in the returned value.
>> >
>> > As an example, given the situation where a user requests events of type
>> > FAN_OPEN. Traditionally, the event mask returned within an event that
>> > occurred on a filesystem object that has been marked for monitoring and is
>> > opened, will only ever have the FAN_OPEN bit set. With the introduction of
>> > the new flags like FAN_OPEN_EXEC, and perhaps any other future event
>> > flags, there is a possibility of the returned event mask containing more
>> > than a single bit set, despite having only requested the single event type.
>> > Prior to these modifications performed to fanotify_should_send_event(), a
>> > user would have received a bundled event mask containing flags FAN_OPEN
>> > and FAN_OPEN_EXEC in the instance that a file was opened for execution via
>> > execve(), for example. This means that a user would receive event types
>> > in the returned event mask that have not been requested. This runs the
>> > possibility of breaking existing systems and causing other unforeseen
>> > issues.
>> >
>> > To mitigate this possibility, fanotify_should_send_event() has been
>> > modified to return the event mask containing ONLY event types explicitly
>> > requested by the user. This means that we will NOT report events that the
>> > user did no set a mask for, and we will NOT report events that the user
>> > has set an ignore mask for.
>> >
>> > The function name fanotify_should_send_event() has also been updated so
>> > that it's more relevant to what it has been designed to do.
>> >
>> > Signed-off-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
>> > Reviewed-by: Amir Goldstein <amir73il@gmail.com>
>> > Signed-off-by: Jan Kara <jack@suse.cz>
>> > Signed-off-by: Sasha Levin <sashal@kernel.org>
>> > ---
>>
>> I have no objection to applying this patch to 4.20, but FYI, it does not
>> fix anything.  Before introducing FAN_OPEN_EXEC in 5.0-rc1, this patch
>> has no visible effect.
>
>Yes, the patch is just a code refactoring useful for the FAN_OPEN_EXEC
>feature.
>
>> I don't mind if you apply it. It will make stable code closer to
>> mainline, which is always a good thing IMO. And FWIW, I think that patch
>> is quite trivial and low risk.
>
>I don't think applying code refactoring to stable is a good idea. Every
>change has a risk of regression and this particular one brings users no
>benefit. So I'd prefer to drop this patch from stable queue.

No objections there, dropping it. Thank you.

--
Thanks,
Sasha

[PATCH AUTOSEL 4.20 110/117] iomap: don't search past page end in iomap_is_partially_uptodate

[Sasha Levin]

From: Eric Sandeen <sandeen@redhat.com>

[ Upstream commit 3cc31fa65d85610574c0f6a474e89f4c419923d5 ]

iomap_is_partially_uptodate() is intended to check wither blocks within
the selected range of a not-uptodate page are uptodate; if the range we
care about is up to date, it's an optimization.

However, the iomap implementation continues to check all blocks up to
from+count, which is beyond the page, and can even be well beyond the
iop->uptodate bitmap.

I think the worst that will happen is that we may eventually find a zero
bit and return "not partially uptodate" when it would have otherwise
returned true, and skip the optimization.  Still, it's clearly an invalid
memory access that must be fixed.

So: fix this by limiting the search to within the page as is done in the
non-iomap variant, block_is_partially_uptodate().

Zorro noticed thiswhen KASAN went off for 512 byte blocks on a 64k
page system:

 BUG: KASAN: slab-out-of-bounds in iomap_is_partially_uptodate+0x1a0/0x1e0
 Read of size 8 at addr ffff800120c3a318 by task fsstress/22337

Reported-by: Zorro Lang <zlang@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 fs/iomap.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/fs/iomap.c b/fs/iomap.c
index d6bc98ae8d35..ce837d962d47 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -492,16 +492,29 @@ iomap_readpages(struct address_space *mapping, struct list_head *pages,
 }
 EXPORT_SYMBOL_GPL(iomap_readpages);
 
+/*
+ * iomap_is_partially_uptodate checks whether blocks within a page are
+ * uptodate or not.
+ *
+ * Returns true if all blocks which correspond to a file portion
+ * we want to read within the page are uptodate.
+ */
 int
 iomap_is_partially_uptodate(struct page *page, unsigned long from,
 		unsigned long count)
 {
 	struct iomap_page *iop = to_iomap_page(page);
 	struct inode *inode = page->mapping->host;
-	unsigned first = from >> inode->i_blkbits;
-	unsigned last = (from + count - 1) >> inode->i_blkbits;
+	unsigned len, first, last;
 	unsigned i;
 
+	/* Limit range to one page */
+	len = min_t(unsigned, PAGE_SIZE - from, count);
+
+	/* First and last blocks in range within page */
+	first = from >> inode->i_blkbits;
+	last = (from + len - 1) >> inode->i_blkbits;
+
 	if (iop) {
 		for (i = first; i <= last; i++)
 			if (!test_bit(i, iop->uptodate))
-- 
2.19.1

[PATCH AUTOSEL 4.20 115/117] userfaultfd: clear flag if remap event not enabled

[Sasha Levin]

From: Peter Xu <peterx@redhat.com>

[ Upstream commit 3cfd22be0ad663248fadfc8f6ffa3e255c394552 ]

When the process being tracked does mremap() without
UFFD_FEATURE_EVENT_REMAP on the corresponding tracking uffd file handle,
we should not generate the remap event, and at the same time we should
clear all the uffd flags on the new VMA.  Without this patch, we can still
have the VM_UFFD_MISSING|VM_UFFD_WP flags on the new VMA even the fault
handling process does not even know the existance of the VMA.

Link: http://lkml.kernel.org/r/20181211053409.20317-1-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com>
Acked-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Reviewed-by: William Kucharski <william.kucharski@oracle.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Kirill A. Shutemov <kirill@shutemov.name>
Cc: Hugh Dickins <hughd@google.com>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Pravin Shedge <pravin.shedge4linux@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 fs/userfaultfd.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 7a85e609fc27..d8b8323e80f4 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -736,10 +736,18 @@ void mremap_userfaultfd_prep(struct vm_area_struct *vma,
 	struct userfaultfd_ctx *ctx;
 
 	ctx = vma->vm_userfaultfd_ctx.ctx;
-	if (ctx && (ctx->features & UFFD_FEATURE_EVENT_REMAP)) {
+
+	if (!ctx)
+		return;
+
+	if (ctx->features & UFFD_FEATURE_EVENT_REMAP) {
 		vm_ctx->ctx = ctx;
 		userfaultfd_ctx_get(ctx);
 		WRITE_ONCE(ctx->mmap_changing, true);
+	} else {
+		/* Drop uffd context if remap feature not enabled */
+		vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
+		vma->vm_flags &= ~(VM_UFFD_WP | VM_UFFD_MISSING);
 	}
 }
 
-- 
2.19.1

[PATCH AUTOSEL 4.20 116/117] mm, proc: be more verbose about unstable VMA flags in /proc/<pid>/smaps

[Sasha Levin]

From: Michal Hocko <mhocko@suse.com>

[ Upstream commit 7550c6079846a24f30d15ac75a941c8515dbedfb ]

Patch series "THP eligibility reporting via proc".

This series of three patches aims at making THP eligibility reporting much
more robust and long term sustainable.  The trigger for the change is a
regression report [2] and the long follow up discussion.  In short the
specific application didn't have good API to query whether a particular
mapping can be backed by THP so it has used VMA flags to workaround that.
These flags represent a deep internal state of VMAs and as such they
should be used by userspace with a great deal of caution.

A similar has happened for [3] when users complained that VM_MIXEDMAP is
no longer set on DAX mappings.  Again a lack of a proper API led to an
abuse.

The first patch in the series tries to emphasise that that the semantic of
flags might change and any application consuming those should be really
careful.

The remaining two patches provide a more suitable interface to address [2]
and provide a consistent API to query the THP status both for each VMA and
process wide as well.  [1]

http://lkml.kernel.org/r/20181120103515.25280-1-mhocko@kernel.org [2]
http://lkml.kernel.org/r/http://lkml.kernel.org/r/alpine.DEB.2.21.1809241054050.224429@chino.kir.corp.google.com
[3] http://lkml.kernel.org/r/20181002100531.GC4135@quack2.suse.cz

This patch (of 3):

Even though vma flags exported via /proc/<pid>/smaps are explicitly
documented to be not guaranteed for future compatibility the warning
doesn't go far enough because it doesn't mention semantic changes to those
flags.  And they are important as well because these flags are a deep
implementation internal to the MM code and the semantic might change at
any time.

Let's consider two recent examples:
http://lkml.kernel.org/r/20181002100531.GC4135@quack2.suse.cz
: commit e1fb4a086495 "dax: remove VM_MIXEDMAP for fsdax and device dax" has
: removed VM_MIXEDMAP flag from DAX VMAs. Now our testing shows that in the
: mean time certain customer of ours started poking into /proc/<pid>/smaps
: and looks at VMA flags there and if VM_MIXEDMAP is missing among the VMA
: flags, the application just fails to start complaining that DAX support is
: missing in the kernel.

http://lkml.kernel.org/r/alpine.DEB.2.21.1809241054050.224429@chino.kir.corp.google.com
: Commit 1860033237d4 ("mm: make PR_SET_THP_DISABLE immediately active")
: introduced a regression in that userspace cannot always determine the set
: of vmas where thp is ineligible.
: Userspace relies on the "nh" flag being emitted as part of /proc/pid/smaps
: to determine if a vma is eligible to be backed by hugepages.
: Previous to this commit, prctl(PR_SET_THP_DISABLE, 1) would cause thp to
: be disabled and emit "nh" as a flag for the corresponding vmas as part of
: /proc/pid/smaps.  After the commit, thp is disabled by means of an mm
: flag and "nh" is not emitted.
: This causes smaps parsing libraries to assume a vma is eligible for thp
: and ends up puzzling the user on why its memory is not backed by thp.

In both cases userspace was relying on a semantic of a specific VMA flag.
The primary reason why that happened is a lack of a proper interface.
While this has been worked on and it will be fixed properly, it seems that
our wording could see some refinement and be more vocal about semantic
aspect of these flags as well.

Link: http://lkml.kernel.org/r/20181211143641.3503-2-mhocko@kernel.org
Signed-off-by: Michal Hocko <mhocko@suse.com>
Acked-by: Jan Kara <jack@suse.cz>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: David Rientjes <rientjes@google.com>
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Paul Oppenheimer <bepvte@gmail.com>
Cc: William Kucharski <william.kucharski@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 Documentation/filesystems/proc.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index 12a5e6e693b6..2a4e63f5122c 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -496,7 +496,9 @@ manner. The codes are the following:
 
 Note that there is no guarantee that every flag and associated mnemonic will
 be present in all further kernel releases. Things get changed, the flags may
-be vanished or the reverse -- new added.
+be vanished or the reverse -- new added. Interpretation of their meaning
+might change in future as well. So each consumer of these flags has to
+follow each specific kernel version for the exact semantic.
 
 This file is only present if the CONFIG_MMU kernel configuration option is
 enabled.
-- 
2.19.1