From: "Jérôme Glisse" <jglisse@redhat.com> To: akpm@linux-foundation.org, <linux-kernel@vger.kernel.org>, linux-mm@kvack.org Cc: "John Hubbard" <jhubbard@nvidia.com>, "Jérôme Glisse" <jglisse@redhat.com> Subject: [HMM v13 14/18] mm/hmm/migrate: support un-addressable ZONE_DEVICE page in migration Date: Fri, 18 Nov 2016 13:18:23 -0500 [thread overview] Message-ID: <1479493107-982-15-git-send-email-jglisse@redhat.com> (raw) In-Reply-To: <1479493107-982-1-git-send-email-jglisse@redhat.com> Allow to unmap and restore special swap entry of un-addressable ZONE_DEVICE memory. Signed-off-by: Jérôme Glisse <jglisse@redhat.com> --- mm/migrate.c | 11 ++++++++++- mm/rmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index 66ce6b4..6b6b457 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -40,6 +40,7 @@ #include <linux/mmu_notifier.h> #include <linux/page_idle.h> #include <linux/page_owner.h> +#include <linux/memremap.h> #include <asm/tlbflush.h> @@ -248,7 +249,15 @@ static int remove_migration_pte(struct page *new, struct vm_area_struct *vma, pte = arch_make_huge_pte(pte, vma, new, 0); } #endif - flush_dcache_page(new); + + if (unlikely(is_zone_device_page(new)) && !is_addressable_page(new)) { + entry = make_device_entry(new, pte_write(pte)); + pte = swp_entry_to_pte(entry); + if (pte_swp_soft_dirty(*ptep)) + pte = pte_mksoft_dirty(pte); + } else + flush_dcache_page(new); + set_pte_at(mm, addr, ptep, pte); if (PageHuge(new)) { diff --git a/mm/rmap.c b/mm/rmap.c index 1ef3640..fff3578 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -61,6 +61,7 @@ #include <linux/hugetlb.h> #include <linux/backing-dev.h> #include <linux/page_idle.h> +#include <linux/memremap.h> #include <asm/tlbflush.h> @@ -1455,6 +1456,52 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma, goto out; } + if ((flags & TTU_MIGRATION) && is_zone_device_page(page)) { + swp_entry_t entry; + pte_t swp_pte; + pmd_t *pmdp; + + if (!(page->pgmap->flags & MEMORY_MOVABLE)) + goto out; + + pmdp = mm_find_pmd(mm, address); + if (!pmdp) + goto out; + + pte = pte_offset_map_lock(mm, pmdp, address, &ptl); + if (!pte) + goto out; + + pteval = ptep_get_and_clear(mm, address, pte); + if (pte_present(pteval) || pte_none(pteval)) { + set_pte_at(mm, address, pte, pteval); + goto out_unmap; + } + + entry = pte_to_swp_entry(pteval); + if (!is_device_entry(entry)) { + set_pte_at(mm, address, pte, pteval); + goto out_unmap; + } + + if (device_entry_to_page(entry) != page) { + set_pte_at(mm, address, pte, pteval); + goto out_unmap; + } + + /* + * Store the pfn of the page in a special migration + * pte. do_swap_page() will wait until the migration + * pte is removed and then restart fault handling. + */ + entry = make_migration_entry(page, 0); + swp_pte = swp_entry_to_pte(entry); + if (pte_soft_dirty(*pte)) + swp_pte = pte_swp_mksoft_dirty(swp_pte); + set_pte_at(mm, address, pte, swp_pte); + goto discard; + } + pte = page_check_address(page, mm, address, &ptl, PageTransCompound(page)); if (!pte) -- 2.4.3
WARNING: multiple messages have this Message-ID (diff)
From: "Jérôme Glisse" <jglisse@redhat.com> To: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "John Hubbard" <jhubbard@nvidia.com>, "Jérôme Glisse" <jglisse@redhat.com> Subject: [HMM v13 14/18] mm/hmm/migrate: support un-addressable ZONE_DEVICE page in migration Date: Fri, 18 Nov 2016 13:18:23 -0500 [thread overview] Message-ID: <1479493107-982-15-git-send-email-jglisse@redhat.com> (raw) In-Reply-To: <1479493107-982-1-git-send-email-jglisse@redhat.com> Allow to unmap and restore special swap entry of un-addressable ZONE_DEVICE memory. Signed-off-by: JA(C)rA'me Glisse <jglisse@redhat.com> --- mm/migrate.c | 11 ++++++++++- mm/rmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index 66ce6b4..6b6b457 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -40,6 +40,7 @@ #include <linux/mmu_notifier.h> #include <linux/page_idle.h> #include <linux/page_owner.h> +#include <linux/memremap.h> #include <asm/tlbflush.h> @@ -248,7 +249,15 @@ static int remove_migration_pte(struct page *new, struct vm_area_struct *vma, pte = arch_make_huge_pte(pte, vma, new, 0); } #endif - flush_dcache_page(new); + + if (unlikely(is_zone_device_page(new)) && !is_addressable_page(new)) { + entry = make_device_entry(new, pte_write(pte)); + pte = swp_entry_to_pte(entry); + if (pte_swp_soft_dirty(*ptep)) + pte = pte_mksoft_dirty(pte); + } else + flush_dcache_page(new); + set_pte_at(mm, addr, ptep, pte); if (PageHuge(new)) { diff --git a/mm/rmap.c b/mm/rmap.c index 1ef3640..fff3578 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -61,6 +61,7 @@ #include <linux/hugetlb.h> #include <linux/backing-dev.h> #include <linux/page_idle.h> +#include <linux/memremap.h> #include <asm/tlbflush.h> @@ -1455,6 +1456,52 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma, goto out; } + if ((flags & TTU_MIGRATION) && is_zone_device_page(page)) { + swp_entry_t entry; + pte_t swp_pte; + pmd_t *pmdp; + + if (!(page->pgmap->flags & MEMORY_MOVABLE)) + goto out; + + pmdp = mm_find_pmd(mm, address); + if (!pmdp) + goto out; + + pte = pte_offset_map_lock(mm, pmdp, address, &ptl); + if (!pte) + goto out; + + pteval = ptep_get_and_clear(mm, address, pte); + if (pte_present(pteval) || pte_none(pteval)) { + set_pte_at(mm, address, pte, pteval); + goto out_unmap; + } + + entry = pte_to_swp_entry(pteval); + if (!is_device_entry(entry)) { + set_pte_at(mm, address, pte, pteval); + goto out_unmap; + } + + if (device_entry_to_page(entry) != page) { + set_pte_at(mm, address, pte, pteval); + goto out_unmap; + } + + /* + * Store the pfn of the page in a special migration + * pte. do_swap_page() will wait until the migration + * pte is removed and then restart fault handling. + */ + entry = make_migration_entry(page, 0); + swp_pte = swp_entry_to_pte(entry); + if (pte_soft_dirty(*pte)) + swp_pte = pte_swp_mksoft_dirty(swp_pte); + set_pte_at(mm, address, pte, swp_pte); + goto discard; + } + pte = page_check_address(page, mm, address, &ptl, PageTransCompound(page)); if (!pte) -- 2.4.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-11-18 17:19 UTC|newest] Thread overview: 146+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-11-18 18:18 [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 01/18] mm/memory/hotplug: convert device parameter bool to set of flags Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 0:44 ` Balbir Singh 2016-11-21 0:44 ` Balbir Singh 2016-11-21 4:53 ` Jerome Glisse 2016-11-21 4:53 ` Jerome Glisse 2016-11-21 6:57 ` Anshuman Khandual 2016-11-21 6:57 ` Anshuman Khandual 2016-11-21 12:19 ` Jerome Glisse 2016-11-21 12:19 ` Jerome Glisse 2016-11-21 6:41 ` Anshuman Khandual 2016-11-21 6:41 ` Anshuman Khandual 2016-11-21 12:27 ` Jerome Glisse 2016-11-21 12:27 ` Jerome Glisse 2016-11-22 5:35 ` Anshuman Khandual 2016-11-22 5:35 ` Anshuman Khandual 2016-11-22 14:08 ` Jerome Glisse 2016-11-22 14:08 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 02/18] mm/ZONE_DEVICE/unaddressable: add support for un-addressable device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 8:06 ` Anshuman Khandual 2016-11-21 8:06 ` Anshuman Khandual 2016-11-21 12:33 ` Jerome Glisse 2016-11-21 12:33 ` Jerome Glisse 2016-11-22 5:15 ` Anshuman Khandual 2016-11-22 5:15 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 03/18] mm/ZONE_DEVICE/free_hot_cold_page: catch ZONE_DEVICE pages Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 8:18 ` Anshuman Khandual 2016-11-21 8:18 ` Anshuman Khandual 2016-11-21 12:50 ` Jerome Glisse 2016-11-21 12:50 ` Jerome Glisse 2016-11-22 4:30 ` Anshuman Khandual 2016-11-22 4:30 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 04/18] mm/ZONE_DEVICE/free-page: callback when page is freed Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 1:49 ` Balbir Singh 2016-11-21 1:49 ` Balbir Singh 2016-11-21 4:57 ` Jerome Glisse 2016-11-21 4:57 ` Jerome Glisse 2016-11-21 8:26 ` Anshuman Khandual 2016-11-21 8:26 ` Anshuman Khandual 2016-11-21 12:34 ` Jerome Glisse 2016-11-21 12:34 ` Jerome Glisse 2016-11-22 5:02 ` Anshuman Khandual 2016-11-22 5:02 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 05/18] mm/ZONE_DEVICE/devmem_pages_remove: allow early removal of device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 10:37 ` Anshuman Khandual 2016-11-21 10:37 ` Anshuman Khandual 2016-11-21 12:39 ` Jerome Glisse 2016-11-21 12:39 ` Jerome Glisse 2016-11-22 4:54 ` Anshuman Khandual 2016-11-22 4:54 ` Anshuman Khandual 2016-11-18 18:18 ` [HMM v13 06/18] mm/ZONE_DEVICE/unaddressable: add special swap for unaddressable Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:06 ` Balbir Singh 2016-11-21 2:06 ` Balbir Singh 2016-11-21 5:05 ` Jerome Glisse 2016-11-21 5:05 ` Jerome Glisse 2016-11-22 2:19 ` Balbir Singh 2016-11-22 2:19 ` Balbir Singh 2016-11-22 13:59 ` Jerome Glisse 2016-11-22 13:59 ` Jerome Glisse 2016-11-21 11:10 ` Anshuman Khandual 2016-11-21 11:10 ` Anshuman Khandual 2016-11-21 10:58 ` Anshuman Khandual 2016-11-21 10:58 ` Anshuman Khandual 2016-11-21 12:42 ` Jerome Glisse 2016-11-21 12:42 ` Jerome Glisse 2016-11-22 4:48 ` Anshuman Khandual 2016-11-22 4:48 ` Anshuman Khandual 2016-11-24 13:56 ` Jerome Glisse 2016-11-24 13:56 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 07/18] mm/ZONE_DEVICE/x86: add support for un-addressable device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:08 ` Balbir Singh 2016-11-21 2:08 ` Balbir Singh 2016-11-21 5:08 ` Jerome Glisse 2016-11-21 5:08 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 08/18] mm/hmm: heterogeneous memory management (HMM for short) Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:29 ` Balbir Singh 2016-11-21 2:29 ` Balbir Singh 2016-11-21 5:14 ` Jerome Glisse 2016-11-21 5:14 ` Jerome Glisse 2016-11-23 4:03 ` Anshuman Khandual 2016-11-23 4:03 ` Anshuman Khandual 2016-11-27 13:10 ` Jerome Glisse 2016-11-27 13:10 ` Jerome Glisse 2016-11-28 2:58 ` Anshuman Khandual 2016-11-28 2:58 ` Anshuman Khandual 2016-11-28 9:41 ` Jerome Glisse 2016-11-28 9:41 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 09/18] mm/hmm/mirror: mirror process address space on device with HMM helpers Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-21 2:42 ` Balbir Singh 2016-11-21 2:42 ` Balbir Singh 2016-11-21 5:18 ` Jerome Glisse 2016-11-21 5:18 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 10/18] mm/hmm/mirror: add range lock helper, prevent CPU page table update for the range Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 11/18] mm/hmm/mirror: add range monitor helper, to monitor CPU page table update Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 12/18] mm/hmm/mirror: helper to snapshot CPU page table Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 13/18] mm/hmm/mirror: device page fault handler Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse [this message] 2016-11-18 18:18 ` [HMM v13 14/18] mm/hmm/migrate: support un-addressable ZONE_DEVICE page in migration Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 15/18] mm/hmm/migrate: add new boolean copy flag to migratepage() callback Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 16/18] mm/hmm/migrate: new memory migration helper for use with device memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 19:57 ` Aneesh Kumar K.V 2016-11-18 19:57 ` Aneesh Kumar K.V 2016-11-18 20:15 ` Jerome Glisse 2016-11-18 20:15 ` Jerome Glisse 2016-11-19 14:32 ` Aneesh Kumar K.V 2016-11-19 14:32 ` Aneesh Kumar K.V 2016-11-19 17:17 ` Jerome Glisse 2016-11-19 17:17 ` Jerome Glisse 2016-11-20 18:21 ` Aneesh Kumar K.V 2016-11-20 18:21 ` Aneesh Kumar K.V 2016-11-20 20:06 ` Jerome Glisse 2016-11-20 20:06 ` Jerome Glisse 2016-11-21 3:30 ` Balbir Singh 2016-11-21 3:30 ` Balbir Singh 2016-11-21 5:31 ` Jerome Glisse 2016-11-21 5:31 ` Jerome Glisse 2016-11-18 18:18 ` [HMM v13 17/18] mm/hmm/devmem: device driver helper to hotplug ZONE_DEVICE memory Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-18 18:18 ` [HMM v13 18/18] mm/hmm/devmem: dummy HMM device as an helper for " Jérôme Glisse 2016-11-18 18:18 ` Jérôme Glisse 2016-11-19 0:41 ` [HMM v13 00/18] HMM (Heterogeneous Memory Management) v13 John Hubbard 2016-11-19 0:41 ` John Hubbard 2016-11-19 14:50 ` Aneesh Kumar K.V 2016-11-19 14:50 ` Aneesh Kumar K.V 2016-11-23 9:16 ` Haggai Eran 2016-11-23 9:16 ` Haggai Eran 2016-11-25 16:16 ` Jerome Glisse 2016-11-25 16:16 ` Jerome Glisse 2016-11-27 13:27 ` Haggai Eran 2016-11-27 13:27 ` Haggai Eran
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1479493107-982-15-git-send-email-jglisse@redhat.com \ --to=jglisse@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=jhubbard@nvidia.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.