From: Sasha Levin <sashal@kernel.org>
To: stable@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: Andrea Arcangeli <aarcange@redhat.com>,
Jerome Glisse <jglisse@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Sasha Levin <sashal@kernel.org>,
linux-mm@kvack.org
Subject: [PATCH AUTOSEL 4.18 33/39] mm: thp: fix MADV_DONTNEED vs migrate_misplaced_transhuge_page race condition
Date: Tue, 13 Nov 2018 00:50:47 -0500 [thread overview]
Message-ID: <20181113055053.78352-33-sashal@kernel.org> (raw)
In-Reply-To: <20181113055053.78352-1-sashal@kernel.org>
From: Andrea Arcangeli <aarcange@redhat.com>
[ Upstream commit d7c3393413fe7e7dc54498ea200ea94742d61e18 ]
Patch series "migrate_misplaced_transhuge_page race conditions".
Aaron found a new instance of the THP MADV_DONTNEED race against
pmdp_clear_flush* variants, that was apparently left unfixed.
While looking into the race found by Aaron, I may have found two more
issues in migrate_misplaced_transhuge_page.
These race conditions would not cause kernel instability, but they'd
corrupt userland data or leave data non zero after MADV_DONTNEED.
I did only minor testing, and I don't expect to be able to reproduce this
(especially the lack of ->invalidate_range before migrate_page_copy,
requires the latest iommu hardware or infiniband to reproduce). The last
patch is noop for x86 and it needs further review from maintainers of
archs that implement flush_cache_range() (not in CC yet).
To avoid confusion, it's not the first patch that introduces the bug fixed
in the second patch, even before removing the
pmdp_huge_clear_flush_notify, that _notify suffix was called after
migrate_page_copy already run.
This patch (of 3):
This is a corollary of ced108037c2aa ("thp: fix MADV_DONTNEED vs. numa
balancing race"), 58ceeb6bec8 ("thp: fix MADV_DONTNEED vs. MADV_FREE
race") and 5b7abeae3af8c ("thp: fix MADV_DONTNEED vs clear soft dirty
race).
When the above three fixes where posted Dave asked
https://lkml.kernel.org/r/929b3844-aec2-0111-fef7-8002f9d4e2b9@intel.com
but apparently this was missed.
The pmdp_clear_flush* in migrate_misplaced_transhuge_page() was introduced
in a54a407fbf7 ("mm: Close races between THP migration and PMD numa
clearing").
The important part of such commit is only the part where the page lock is
not released until the first do_huge_pmd_numa_page() finished disarming
the pagenuma/protnone.
The addition of pmdp_clear_flush() wasn't beneficial to such commit and
there's no commentary about such an addition either.
I guess the pmdp_clear_flush() in such commit was added just in case for
safety, but it ended up introducing the MADV_DONTNEED race condition found
by Aaron.
At that point in time nobody thought of such kind of MADV_DONTNEED race
conditions yet (they were fixed later) so the code may have looked more
robust by adding the pmdp_clear_flush().
This specific race condition won't destabilize the kernel, but it can
confuse userland because after MADV_DONTNEED the memory won't be zeroed
out.
This also optimizes the code and removes a superfluous TLB flush.
[akpm@linux-foundation.org: reflow comment to 80 cols, fix grammar and typo (beacuse)]
Link: http://lkml.kernel.org/r/20181013002430.698-2-aarcange@redhat.com
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Reported-by: Aaron Tomlin <atomlin@redhat.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
mm/migrate.c | 25 ++++++++++++++++++-------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/mm/migrate.c b/mm/migrate.c
index f49eb9589d73..38ad6365ed10 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2086,15 +2086,26 @@ int migrate_misplaced_transhuge_page(struct mm_struct *mm,
entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
/*
- * Clear the old entry under pagetable lock and establish the new PTE.
- * Any parallel GUP will either observe the old page blocking on the
- * page lock, block on the page table lock or observe the new page.
- * The SetPageUptodate on the new page and page_add_new_anon_rmap
- * guarantee the copy is visible before the pagetable update.
+ * Overwrite the old entry under pagetable lock and establish
+ * the new PTE. Any parallel GUP will either observe the old
+ * page blocking on the page lock, block on the page table
+ * lock or observe the new page. The SetPageUptodate on the
+ * new page and page_add_new_anon_rmap guarantee the copy is
+ * visible before the pagetable update.
*/
flush_cache_range(vma, mmun_start, mmun_end);
page_add_anon_rmap(new_page, vma, mmun_start, true);
- pmdp_huge_clear_flush_notify(vma, mmun_start, pmd);
+ /*
+ * At this point the pmd is numa/protnone (i.e. non present) and the TLB
+ * has already been flushed globally. So no TLB can be currently
+ * caching this non present pmd mapping. There's no need to clear the
+ * pmd before doing set_pmd_at(), nor to flush the TLB after
+ * set_pmd_at(). Clearing the pmd here would introduce a race
+ * condition against MADV_DONTNEED, because MADV_DONTNEED only holds the
+ * mmap_sem for reading. If the pmd is set to NULL at any given time,
+ * MADV_DONTNEED won't wait on the pmd lock and it'll skip clearing this
+ * pmd.
+ */
set_pmd_at(mm, mmun_start, pmd, entry);
update_mmu_cache_pmd(vma, address, &entry);
@@ -2108,7 +2119,7 @@ int migrate_misplaced_transhuge_page(struct mm_struct *mm,
* No need to double call mmu_notifier->invalidate_range() callback as
* the above pmdp_huge_clear_flush_notify() did already call it.
*/
- mmu_notifier_invalidate_range_only_end(mm, mmun_start, mmun_end);
+ mmu_notifier_invalidate_range_end(mm, mmun_start, mmun_end);
/* Take an "isolate" reference and put new page on the LRU. */
get_page(new_page);
--
2.17.1
next prev parent reply other threads:[~2018-11-13 5:51 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-13 5:50 [PATCH AUTOSEL 4.18 01/39] bfs: add sanity check at bfs_fill_super() Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 02/39] cifs: don't dereference smb_file_target before null check Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 03/39] cifs: fix return value for cifs_listxattr Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 04/39] arm64: kprobe: make page to RO mode when allocate it Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 05/39] nvme-pci: fix conflicting p2p resource adds Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 06/39] block: brd: associate with queue until adding disk Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 07/39] net: hns3: bugfix for rtnl_lock's range in the hclgevf_reset() Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 08/39] net: hns3: bugfix for rtnl_lock's range in the hclge_reset() Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 09/39] net: hns3: bugfix for the initialization of command queue's spin lock Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 10/39] ixgbe: fix MAC anti-spoofing filter after VFLR Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 11/39] mm: Fix warning in insert_pfn() Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 12/39] mm/memory_hotplug: make add_memory() take the device_hotplug_lock Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 13/39] reiserfs: propagate errors from fill_with_dentries() properly Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 14/39] hfs: prevent btree data loss on root split Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 15/39] hfsplus: " Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 16/39] mm/gup_benchmark.c: prevent integer overflow in ioctl Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 17/39] perf unwind: Take pgoff into account when reporting elf to libdwfl Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 18/39] um: Give start_idle_thread() a return code Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 19/39] drm/edid: Add 6 bpc quirk for BOE panel Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 20/39] afs: Handle EIO from delivery function Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 21/39] platform/x86: intel_telemetry: report debugfs failure Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 22/39] clk: fixed-rate: fix of_node_get-put imbalance Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 23/39] perf symbols: Set PLT entry/header sizes properly on Sparc Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 24/39] fs/exofs: fix potential memory leak in mount option parsing Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 25/39] clk: samsung: exynos5420: Enable PERIS clocks for suspend Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 26/39] apparmor: Fix uninitialized value in aa_split_fqname Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 27/39] x86/earlyprintk: Add a force option for pciserial device Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 28/39] platform/x86: acerhdf: Add BIOS entry for Gateway LT31 v1.3307 Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 29/39] arm64: percpu: Initialize ret in the default case Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 30/39] clk: meson: clk-pll: drop CLK_GET_RATE_NOCACHE where unnecessary Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 31/39] clk: sunxi-ng: sun50i: h6: Add 2x fixed post-divider to MMC module clocks Sasha Levin
2018-11-13 12:27 ` Icenowy Zheng
2018-11-22 19:35 ` Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 32/39] clk: ti: fix OF child-node lookup Sasha Levin
2018-11-13 5:50 ` Sasha Levin [this message]
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 34/39] mm: thp: fix mmu_notifier in migrate_misplaced_transhuge_page() Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 35/39] mm: calculate deferred pages after skipping mirrored memory Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 36/39] mm/vmstat.c: assert that vmstat_text is in sync with stat_items_size Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 37/39] userfaultfd: allow get_mempolicy(MPOL_F_NODE|MPOL_F_ADDR) to trigger userfaults Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 38/39] mm: don't miss the last page because of round-off error Sasha Levin
2018-11-13 5:50 ` [PATCH AUTOSEL 4.18 39/39] mm: don't warn about large allocations for slab Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181113055053.78352-33-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=jglisse@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).