[PATCH v3 0/3] mm/hmm: fixes for device private page migration

[PATCH v3 0/3] mm/hmm: fixes for device private page migration

[Ralph Campbell]

Testing the latest linux git tree turned up a few bugs with page
migration to and from ZONE_DEVICE private and anonymous pages.
Hopefully this series clarifies how ZONE_DEVICE private struct page
uses the same mapping and index fields from the source anonymous page
mapping.

Changes from v2 to v3:

Patch #1 is basically new (like v1 but with comments from v2) to
accommodate Matthew Wilcox's NAK of v2 and Christoph's objection to
adding _zd_pad fields in v1.

Patch #2 adds reviewed-by

Patch #3 adds comments explaining the reason for setting "subpage".

Changes from v1 to v2:

Patch #1 merges ZONE_DEVICE page struct into a union of lru and
a struct for ZONE_DEVICE fields. So, basically a new patch.

Patch #2 updates the code comments for clearing page->mapping as
suggested by John Hubbard.

Patch #3 is unchanged from the previous posting but note that
Andrew Morton has v1 queued in v5.2-mmotm-2019-07-18-16-08.

Ralph Campbell (3):
  mm: document zone device struct page reserved fields
  mm/hmm: fix ZONE_DEVICE anon page mapping reuse
  mm/hmm: Fix bad subpage pointer in try_to_unmap_one

 include/linux/mm_types.h | 9 ++++++++-
 kernel/memremap.c        | 4 ++++
 mm/rmap.c                | 1 +
 3 files changed, 13 insertions(+), 1 deletion(-)

-- 
2.20.1

[PATCH v3 1/3] mm: document zone device struct page field usage

[Ralph Campbell]

Struct page for ZONE_DEVICE private pages uses the page->mapping and
and page->index fields while the source anonymous pages are migrated to
device private memory. This is so rmap_walk() can find the page when
migrating the ZONE_DEVICE private page back to system memory.
ZONE_DEVICE pmem backed fsdax pages also use the page->mapping and
page->index fields when files are mapped into a process address space.

Add comments to struct page and remove the unused "_zd_pad_1" field
to make this more clear.

Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Christoph Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Jérôme Glisse <jglisse@redhat.com>
Cc: "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jason Gunthorpe <jgg@mellanox.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
---
 include/linux/mm_types.h | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 3a37a89eb7a7..6a7a1083b6fb 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -159,7 +159,16 @@ struct page {
 			/** @pgmap: Points to the hosting device page map. */
 			struct dev_pagemap *pgmap;
 			void *zone_device_data;
-			unsigned long _zd_pad_1;	/* uses mapping */
+			/*
+			 * ZONE_DEVICE private pages are counted as being
+			 * mapped so the next 3 words hold the mapping, index,
+			 * and private fields from the source anonymous or
+			 * page cache page while the page is migrated to device
+			 * private memory.
+			 * ZONE_DEVICE MEMORY_DEVICE_FS_DAX pages also
+			 * use the mapping, index, and private fields when
+			 * pmem backed DAX files are mapped.
+			 */
 		};
 
 		/** @rcu_head: You can use this to free a page by RCU. */
-- 
2.20.1

Re: [PATCH v3 1/3] mm: document zone device struct page field usage

[Jason Gunthorpe]

On Wed, Jul 24, 2019 at 04:26:58PM -0700, Ralph Campbell wrote:
> Struct page for ZONE_DEVICE private pages uses the page->mapping and
> and page->index fields while the source anonymous pages are migrated to
> device private memory. This is so rmap_walk() can find the page when
> migrating the ZONE_DEVICE private page back to system memory.
> ZONE_DEVICE pmem backed fsdax pages also use the page->mapping and
> page->index fields when files are mapped into a process address space.
> 
> Add comments to struct page and remove the unused "_zd_pad_1" field
> to make this more clear.
> 
> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
> Reviewed-by: John Hubbard <jhubbard@nvidia.com>
> Cc: Matthew Wilcox <willy@infradead.org>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: Christoph Lameter <cl@linux.com>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Jérôme Glisse <jglisse@redhat.com>
> Cc: "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
> Cc: Lai Jiangshan <jiangshanlai@gmail.com>
> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Cc: Pekka Enberg <penberg@kernel.org>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Jason Gunthorpe <jgg@mellanox.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
>  include/linux/mm_types.h | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)

Ralph, you marked some of thes patches as mm/hmm, but I feel it is
best if Andrew takes them through the normal -mm path.

They don't touch hmm.c or mmu notifiers so I don't forsee conflicts,
and I don't feel comfortable to review this code.

Regards,
Jason

Re: [PATCH v3 1/3] mm: document zone device struct page field usage

[Ralph Campbell]

On 7/24/19 6:22 PM, Jason Gunthorpe wrote:
> On Wed, Jul 24, 2019 at 04:26:58PM -0700, Ralph Campbell wrote:
>> Struct page for ZONE_DEVICE private pages uses the page->mapping and
>> and page->index fields while the source anonymous pages are migrated to
>> device private memory. This is so rmap_walk() can find the page when
>> migrating the ZONE_DEVICE private page back to system memory.
>> ZONE_DEVICE pmem backed fsdax pages also use the page->mapping and
>> page->index fields when files are mapped into a process address space.
>>
>> Add comments to struct page and remove the unused "_zd_pad_1" field
>> to make this more clear.
>>
>> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
>> Reviewed-by: John Hubbard <jhubbard@nvidia.com>
>> Cc: Matthew Wilcox <willy@infradead.org>
>> Cc: Vlastimil Babka <vbabka@suse.cz>
>> Cc: Christoph Lameter <cl@linux.com>
>> Cc: Dave Hansen <dave.hansen@linux.intel.com>
>> Cc: Jérôme Glisse <jglisse@redhat.com>
>> Cc: "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
>> Cc: Lai Jiangshan <jiangshanlai@gmail.com>
>> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
>> Cc: Pekka Enberg <penberg@kernel.org>
>> Cc: Randy Dunlap <rdunlap@infradead.org>
>> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
>> Cc: Christoph Hellwig <hch@lst.de>
>> Cc: Jason Gunthorpe <jgg@mellanox.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: Linus Torvalds <torvalds@linux-foundation.org>
>>   include/linux/mm_types.h | 11 ++++++++++-
>>   1 file changed, 10 insertions(+), 1 deletion(-)
> 
> Ralph, you marked some of thes patches as mm/hmm, but I feel it is
> best if Andrew takes them through the normal -mm path.
> 
> They don't touch hmm.c or mmu notifiers so I don't forsee conflicts,
> and I don't feel comfortable to review this code.
> 
> Regards,
> Jason
> 

Fine with me. I should have been clear in the cover letter which
tree to target.

Re: [PATCH v3 1/3] mm: document zone device struct page field usage

[Christoph Hellwig]

On Wed, Jul 24, 2019 at 04:26:58PM -0700, Ralph Campbell wrote:
> Struct page for ZONE_DEVICE private pages uses the page->mapping and
> and page->index fields while the source anonymous pages are migrated to
> device private memory. This is so rmap_walk() can find the page when
> migrating the ZONE_DEVICE private page back to system memory.
> ZONE_DEVICE pmem backed fsdax pages also use the page->mapping and
> page->index fields when files are mapped into a process address space.
> 
> Add comments to struct page and remove the unused "_zd_pad_1" field
> to make this more clear.

I still think we should also fix up the layout, and I haven't seen
a reply from Matthew justifying his curses for your patch that makes
the struct page layout actually match how it is used.

Re: [PATCH v3 1/3] mm: document zone device struct page field usage

[Ralph Campbell]

On 7/24/19 10:38 PM, Christoph Hellwig wrote:
> On Wed, Jul 24, 2019 at 04:26:58PM -0700, Ralph Campbell wrote:
>> Struct page for ZONE_DEVICE private pages uses the page->mapping and
>> and page->index fields while the source anonymous pages are migrated to
>> device private memory. This is so rmap_walk() can find the page when
>> migrating the ZONE_DEVICE private page back to system memory.
>> ZONE_DEVICE pmem backed fsdax pages also use the page->mapping and
>> page->index fields when files are mapped into a process address space.
>>
>> Add comments to struct page and remove the unused "_zd_pad_1" field
>> to make this more clear.
> 
> I still think we should also fix up the layout, and I haven't seen
> a reply from Matthew justifying his curses for your patch that makes
> the struct page layout actually match how it is used.
> 

Well, I can kind of see this both ways since ZONE_DEVICE
MEMORY_DEVICE_DEVDAX and MEMORY_DEVICE_PCI_P2PDMA don't
seem to use the 3 words like MEMORY_DEVICE_PRIVATE and
MEMORY_DEVICE_FS_DAX.

I like v3 because not all of the ZONE_DEVICE types are handled
the same in regards to using the 3 words and there may be future
ZONE_DEVICE types that use the 3 words differently which might
require a union.

I agree, I would like to hear from Matthew on his thoughts.

[PATCH v3 2/3] mm/hmm: fix ZONE_DEVICE anon page mapping reuse

[Ralph Campbell]

When a ZONE_DEVICE private page is freed, the page->mapping field can be
set. If this page is reused as an anonymous page, the previous value can
prevent the page from being inserted into the CPU's anon rmap table.
For example, when migrating a pte_none() page to device memory:
  migrate_vma(ops, vma, start, end, src, dst, private)
    migrate_vma_collect()
      src[] = MIGRATE_PFN_MIGRATE
    migrate_vma_prepare()
      /* no page to lock or isolate so OK */
    migrate_vma_unmap()
      /* no page to unmap so OK */
    ops->alloc_and_copy()
      /* driver allocates ZONE_DEVICE page for dst[] */
    migrate_vma_pages()
      migrate_vma_insert_page()
        page_add_new_anon_rmap()
          __page_set_anon_rmap()
            /* This check sees the page's stale mapping field */
            if (PageAnon(page))
              return
            /* page->mapping is not updated */

The result is that the migration appears to succeed but a subsequent CPU
fault will be unable to migrate the page back to system memory or worse.

Clear the page->mapping field when freeing the ZONE_DEVICE page so stale
pointer data doesn't affect future page use.

Fixes: b7a523109fb5c9d2d6dd ("mm: don't clear ->mapping in hmm_devmem_free")
Cc: stable@vger.kernel.org
Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Jason Gunthorpe <jgg@mellanox.com>
Cc: Logan Gunthorpe <logang@deltatee.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Jan Kara <jack@suse.cz>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
---
 kernel/memremap.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/kernel/memremap.c b/kernel/memremap.c
index 6ee03a816d67..289a086e1467 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -397,6 +397,30 @@ void __put_devmap_managed_page(struct page *page)
 
 		mem_cgroup_uncharge(page);
 
+		/*
+		 * When a device_private page is freed, the page->mapping field
+		 * may still contain a (stale) mapping value. For example, the
+		 * lower bits of page->mapping may still identify the page as
+		 * an anonymous page. Ultimately, this entire field is just
+		 * stale and wrong, and it will cause errors if not cleared.
+		 * One example is:
+		 *
+		 *  migrate_vma_pages()
+		 *    migrate_vma_insert_page()
+		 *      page_add_new_anon_rmap()
+		 *        __page_set_anon_rmap()
+		 *          ...checks page->mapping, via PageAnon(page) call,
+		 *            and incorrectly concludes that the page is an
+		 *            anonymous page. Therefore, it incorrectly,
+		 *            silently fails to set up the new anon rmap.
+		 *
+		 * For other types of ZONE_DEVICE pages, migration is either
+		 * handled differently or not done at all, so there is no need
+		 * to clear page->mapping.
+		 */
+		if (is_device_private_page(page))
+			page->mapping = NULL;
+
 		page->pgmap->ops->page_free(page);
 	} else if (!count)
 		__put_page(page);
-- 
2.20.1

Re: [PATCH v3 2/3] mm/hmm: fix ZONE_DEVICE anon page mapping reuse

[Christoph Hellwig]

Andrew,

Any reason the fixes haven't made it to mainline yet?

Re: [PATCH v3 2/3] mm/hmm: fix ZONE_DEVICE anon page mapping reuse

[Andrew Morton]

On Fri, 2 Aug 2019 10:31:41 +0200 Christoph Hellwig <hch@lst.de> wrote:

> Andrew,
> 
> Any reason the fixes haven't made it to mainline yet?

I generally let fixes bake in -next for a week or two, unless they
appear to be impeding ongoing test&devel.

[3/3] shows no sign of having been reviewed, btw.

[PATCH v3 3/3] mm/hmm: Fix bad subpage pointer in try_to_unmap_one

[Ralph Campbell]

When migrating an anonymous private page to a ZONE_DEVICE private page,
the source page->mapping and page->index fields are copied to the
destination ZONE_DEVICE struct page and the page_mapcount() is increased.
This is so rmap_walk() can be used to unmap and migrate the page back to
system memory. However, try_to_unmap_one() computes the subpage pointer
from a swap pte which computes an invalid page pointer and a kernel panic
results such as:

BUG: unable to handle page fault for address: ffffea1fffffffc8

Currently, only single pages can be migrated to device private memory so
no subpage computation is needed and it can be set to "page".

Fixes: a5430dda8a3a1c ("mm/migrate: support un-addressable ZONE_DEVICE page in migration")
Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jason Gunthorpe <jgg@mellanox.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 mm/rmap.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/mm/rmap.c b/mm/rmap.c
index e5dfe2ae6b0d..003377e24232 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1475,7 +1475,15 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 			/*
 			 * No need to invalidate here it will synchronize on
 			 * against the special swap migration pte.
+			 *
+			 * The assignment to subpage above was computed from a
+			 * swap PTE which results in an invalid pointer.
+			 * Since only PAGE_SIZE pages can currently be
+			 * migrated, just set it to page. This will need to be
+			 * changed when hugepage migrations to device private
+			 * memory are supported.
 			 */
+			subpage = page;
 			goto discard;
 		}
 
-- 
2.20.1

Re: [PATCH v3 3/3] mm/hmm: Fix bad subpage pointer in try_to_unmap_one

[Sasha Levin]

Hi,

[This is an automated email]

This commit has been processed because it contains a "Fixes:" tag,
fixing commit: a5430dda8a3a mm/migrate: support un-addressable ZONE_DEVICE page in migration.

The bot has tested the following trees: v5.2.2, v5.1.19, v4.19.60, v4.14.134.

v5.2.2: Build OK!
v5.1.19: Build OK!
v4.19.60: Build OK!
v4.14.134: Failed to apply! Possible dependencies:
    0f10851ea475 ("mm/mmu_notifier: avoid double notification when it is useless")


NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

--
Thanks,
Sasha