mm-commits Archive on lore.kernel.org
 help / color / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, catalin.marinas@arm.com,
	hannes@cmpxchg.org, hdanton@sina.com, hughd@google.com,
	josef@toxicpanda.com, kirill.shutemov@linux.intel.com,
	linux-mm@kvack.org, mm-commits@vger.kernel.org,
	torvalds@linux-foundation.org, will.deacon@arm.com,
	willy@infradead.org, xuyu@linux.alibaba.com,
	yang.shi@linux.alibaba.com
Subject: [patch 01/15] mm/memory.c: avoid access flag update TLB flush for retried page fault
Date: Thu, 23 Jul 2020 21:15:08 -0700
Message-ID: <20200724041508.QlTbrHnfh%akpm@linux-foundation.org> (raw)
In-Reply-To: <20200723211432.b31831a0df3bc2cbdae31b40@linux-foundation.org>

From: Yang Shi <yang.shi@linux.alibaba.com>
Subject: mm/memory.c: avoid access flag update TLB flush for retried page fault

Recently we found regression when running will_it_scale/page_fault3 test
on ARM64.  Over 70% down for the multi processes cases and over 20% down
for the multi threads cases.  It turns out the regression is caused by
commit 89b15332af7c0312a41e50846819ca6613b58b4c ("mm: drop mmap_sem before
calling balance_dirty_pages() in write fault").

The test mmaps a memory size file then write to the mapping, this would
make all memory dirty and trigger dirty pages throttle, that upstream
commit would release mmap_sem then retry the page fault.  The retried page
fault would see correct PTEs installed by the first try then update dirty
bit and clear read-only bit and flush TLBs for ARM.  The regression is
caused by the excessive TLB flush.  It is fine on x86 since x86 doesn't
clear read-only bit so there is no need to flush TLB for this case.

The page fault would be retried due to:
1. Waiting for page readahead
2. Waiting for page swapped in
3. Waiting for dirty pages throttling

The first two cases don't have PTEs set up at all, so the retried page
fault would install the PTEs, so they don't reach there.  But the #3 case
usually has PTEs installed, the retried page fault would reach the dirty
bit and read-only bit update.  But it seems not necessary to modify those
bits again for #3 since they should be already set by the first page fault
try.

Of course the parallel page fault may set up PTEs, but we just need care
about write fault.  If the parallel page fault setup a writable and dirty
PTE then the retried fault doesn't need do anything extra.  If the
parallel page fault setup a clean read-only PTE, the retried fault should
just call do_wp_page() then return as the below code snippet shows:

if (vmf->flags & FAULT_FLAG_WRITE) {
        if (!pte_write(entry))
            return do_wp_page(vmf);
}

With this fix the test result get back to normal.

Link: http://lkml.kernel.org/r/1594148072-91273-1-git-send-email-yang.shi@linux.alibaba.com
Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
Reported-by: Xu Yu <xuyu@linux.alibaba.com>
Debugged-by: Xu Yu <xuyu@linux.alibaba.com>
Tested-by: Xu Yu <xuyu@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Josef Bacik <josef@toxicpanda.com>
Cc: Hillf Danton <hdanton@sina.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 mm/memory.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

--- a/mm/memory.c~mm-avoid-access-flag-update-tlb-flush-for-retried-page-fault
+++ a/mm/memory.c
@@ -4241,8 +4241,13 @@ static vm_fault_t handle_pte_fault(struc
 	if (vmf->flags & FAULT_FLAG_WRITE) {
 		if (!pte_write(entry))
 			return do_wp_page(vmf);
-		entry = pte_mkdirty(entry);
 	}
+
+	if ((vmf->flags & FAULT_FLAG_WRITE) && !(vmf->flags & FAULT_FLAG_TRIED))
+		entry = pte_mkdirty(entry);
+	else if (vmf->flags & FAULT_FLAG_TRIED)
+		goto unlock;
+
 	entry = pte_mkyoung(entry);
 	if (ptep_set_access_flags(vmf->vma, vmf->address, vmf->pte, entry,
 				vmf->flags & FAULT_FLAG_WRITE)) {
_

  reply index

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-24  4:14 incoming Andrew Morton
2020-07-24  4:15 ` Andrew Morton [this message]
     [not found]   ` <30cf7356-bef1-c621-60cb-e12a8bd9111d@linux.alibaba.com>
2020-07-24  4:56     ` [patch 01/15] mm/memory.c: avoid access flag update TLB flush for retried page fault Andrew Morton
2020-07-24  4:15 ` [patch 02/15] mm/mmap.c: close race between munmap() and expand_upwards()/downwards() Andrew Morton
2020-07-24  4:15 ` [patch 03/15] vfs/xattr: mm/shmem: kernfs: release simple xattr entry in a right way Andrew Morton
2020-07-24  4:15 ` [patch 04/15] mm: initialize return of vm_insert_pages Andrew Morton
2020-07-24  4:15 ` [patch 05/15] mm/memcontrol: fix OOPS inside mem_cgroup_get_nr_swap_pages() Andrew Morton
2020-07-24  4:15 ` [patch 06/15] mm/memcg: fix refcount error while moving and swapping Andrew Morton
2020-07-24  4:15 ` [patch 07/15] mm: memcg/slab: fix memory leak at non-root kmem_cache destroy Andrew Morton
2020-07-24  4:15 ` [patch 08/15] mm/hugetlb: avoid hardcoding while checking if cma is enabled Andrew Morton
2020-07-24  4:15 ` [patch 09/15] khugepaged: fix null-pointer dereference due to race Andrew Morton
2020-07-24  4:15 ` [patch 10/15] mailmap: add entry for Mike Rapoport Andrew Morton
2020-07-24  4:15 ` [patch 11/15] squashfs: fix length field overlap check in metadata reading Andrew Morton
2020-07-24  4:15 ` [patch 12/15] scripts/decode_stacktrace: strip basepath from all paths Andrew Morton
2020-07-24  4:15 ` [patch 13/15] io-mapping: indicate mapping failure Andrew Morton
2020-07-24  4:15 ` [patch 14/15] MAINTAINERS: add KCOV section Andrew Morton
2020-07-24  4:15 ` [patch 15/15] scripts/gdb: fix lx-symbols 'gdb.error' while loading modules Andrew Morton
2020-07-27 19:47 ` + mm-remove-unnecessary-wrapper-function-do_mmap_pgoff.patch added to -mm tree Andrew Morton
2020-07-27 19:56 ` + nilfs2-only-call-unlock_new_inode-if-i_new.patch " Andrew Morton
2020-07-27 19:57 ` + nilfs2-convert-__nilfs_msg-to-integrate-the-level-and-format.patch " Andrew Morton
2020-07-27 19:57 ` + nilfs2-use-a-more-common-logging-style.patch " Andrew Morton
2020-07-27 19:58 ` + checkpatch-add-test-for-repeated-words.patch " Andrew Morton
2020-07-27 20:05 ` + ocfs2-replace-http-links-with-https-ones.patch " Andrew Morton
2020-07-27 20:09 ` + ocfs2-fix-unbalanced-locking.patch " Andrew Morton
2020-07-27 20:10 ` + kernelh-remove-duplicate-include-of-asm-div64h.patch " Andrew Morton
2020-07-27 20:11 ` + tools-replace-http-links-with-https-ones.patch " Andrew Morton
2020-07-27 20:12 ` + lib-replace-http-links-with-https-ones.patch " Andrew Morton
2020-07-27 20:12 ` + include-replace-http-links-with-https-ones.patch " Andrew Morton
2020-07-27 20:34 ` + mm-make-mm-locked_vm-an-atomic64-counter.patch " Andrew Morton
2020-07-27 20:34 ` + mm-util-account_locked_vm-does-not-hold-mmap_lock.patch " Andrew Morton
2020-07-27 20:37 ` + cg_read_strcmp-fix-null-pointer-dereference.patch " Andrew Morton
2020-07-27 20:51 ` + mm-hugetlb-add-mempolicy-check-in-the-reservation-routine.patch " Andrew Morton
2020-07-27 20:52 ` [withdrawn] checkpatch-support-deprecated-terms-checking.patch removed from " Andrew Morton
2020-07-27 23:47 ` [obsolete] scripts-deprecated_terms-recommend-denylist-allowlist-instead-of-blacklist-whitelist.patch " Andrew Morton
2020-07-27 23:50 ` [obsolete] scripts-deprecated_terms-sync-with-inclusive-terms.patch " Andrew Morton
2020-07-28  0:18 ` [failures] mm-hugetlb-add-mempolicy-check-in-the-reservation-routine.patch " Andrew Morton
2020-07-28  1:19 ` mmotm 2020-07-27-18-18 uploaded Andrew Morton
     [not found]   ` <ae87385b-f830-dbdf-ebc7-1afb82a7fed0@infradead.org>
2020-07-28 21:55     ` mmotm 2020-07-27-18-18 uploaded (mm/page_alloc.c) Andrew Morton
     [not found]       ` <20200729082053.6c2fb654@canb.auug.org.au>
2020-07-28 22:31         ` Andrew Morton
     [not found]       ` <048cef07-ad4b-8788-94a4-e144de731ab6@infradead.org>
2020-07-29  1:44         ` Andrew Morton
2020-07-28 20:53 ` + mm-mempolicy-fix-kerneldoc-of-numa_map_to_online_node.patch added to -mm tree Andrew Morton
2020-07-28 20:53 ` + mm-mmu_notifier-fix-and-extend-kerneldoc.patch " Andrew Morton
2020-07-28 20:54 ` + mm-swap-fix-kerneldoc-of-swap_vma_readahead.patch " Andrew Morton
2020-07-28 20:58 ` + mm-memcontrol-dont-count-limit-setting-reclaim-as-memory-pressure.patch " Andrew Morton
2020-07-28 21:01 ` + mm-memcontrol-restore-proper-dirty-throttling-when-memoryhigh-changes.patch " Andrew Morton
2020-07-28 22:06 ` + mm-compaction-correct-the-comments-of-compact_defer_shift.patch " Andrew Morton
2020-07-28 22:09 ` + selftests-add-mincore-tests.patch " Andrew Morton
2020-07-28 22:16 ` + proc-pid-smaps-consistent-whitespace-output-format.patch " Andrew Morton
2020-07-28 22:21 ` + xtensa-switch-to-generic-version-of-pte-allocation-fix.patch " Andrew Morton
2020-07-28 22:21 ` Andrew Morton
2020-07-29 21:49 ` + mm-slab-avoid-the-use-of-one-element-array-and-use-struct_size-helper.patch " Andrew Morton
2020-07-29 23:52 ` [obsolete] mm-slab-avoid-the-use-of-one-element-array-and-use-struct_size-helper.patch removed from " Andrew Morton
2020-07-31 19:24 ` + kasan-dont-tag-stacks-allocated-with-pagealloc.patch added to " Andrew Morton
2020-07-31 19:24 ` + kasan-arm64-dont-instrument-functions-that-enable-kasan.patch " Andrew Morton
2020-07-31 19:24 ` + kasan-allow-enabling-stack-tagging-for-tag-based-mode.patch " Andrew Morton
2020-07-31 19:24 ` + kasan-adjust-kasan_stack_oob-for-tag-based-mode.patch " Andrew Morton
2020-07-31 20:00 ` [obsolete] mmhwpoison-rework-soft-offline-for-in-use-pages-fix.patch removed from " Andrew Morton
2020-07-31 20:05 ` + mmhwpoison-cleanup-unused-pagehuge-check.patch added to " Andrew Morton
2020-07-31 20:05 ` + mm-hwpoison-remove-recalculating-hpage.patch " Andrew Morton
2020-07-31 20:05 ` + mmmadvise-call-soft_offline_page-without-mf_count_increased.patch " Andrew Morton
2020-07-31 20:05 ` + mmmadvise-refactor-madvise_inject_error.patch " Andrew Morton
2020-07-31 20:05 ` + mmhwpoison-inject-dont-pin-for-hwpoison_filter.patch " Andrew Morton
2020-07-31 20:05 ` + mmhwpoison-un-export-get_hwpoison_page-and-make-it-static.patch " Andrew Morton
2020-07-31 20:05 ` + mmhwpoison-kill-put_hwpoison_page.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-remove-mf_count_increased.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-remove-flag-argument-from-soft-offline-functions.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-unify-thp-handling-for-hard-and-soft-offline.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-rework-soft-offline-for-free-pages.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-rework-soft-offline-for-in-use-pages.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-refactor-soft_offline_huge_page-and-__soft_offline_page.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-return-0-if-the-page-is-already-poisoned-in-soft-offline.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-introduce-mf_msg_unsplit_thp.patch " Andrew Morton
2020-07-31 20:06 ` + mmhwpoison-double-check-page-count-in-__get_any_page.patch " Andrew Morton
2020-07-31 20:23 ` + mm-gup-restrict-cma-region-by-using-allocation-scope-api.patch " Andrew Morton
2020-07-31 20:23 ` + mm-hugetlb-make-hugetlb-migration-callback-cma-aware.patch " Andrew Morton
2020-07-31 20:23 ` + mm-gup-use-a-standard-migration-target-allocation-callback.patch " Andrew Morton
2020-07-31 20:25 ` + mm-migrate-make-a-standard-migration-target-allocation-function-fix.patch " Andrew Morton
2020-07-31 20:26 ` + mm-memcontrol-decouple-reference-counting-from-page-accounting-fix.patch " Andrew Morton
2020-07-31 20:32 ` + mm-dmapoolc-add-warn_on-in-dma_pool_destroy.patch " Andrew Morton
2020-07-31 20:49 ` + kstrto-correct-documentation-references-to-simple_strto.patch " Andrew Morton
2020-07-31 20:49 ` + kstrto-do-not-describe-simple_strto-as-obsolete-replaced.patch " Andrew Morton
2020-07-31 20:57 ` + mm-hugetlb-fix-calculation-of-adjust_range_if_pmd_sharing_possible.patch " Andrew Morton
2020-07-31 20:59 ` + poison-remove-obsolete-comment.patch " Andrew Morton
2020-07-31 21:02 ` + cma-dont-quit-at-first-error-when-activating-reserved-areas.patch " Andrew Morton
2020-07-31 21:10 ` [nacked] mm-dmapoolc-add-warn_on-in-dma_pool_destroy.patch removed from " Andrew Morton
2020-07-31 23:46 ` mmotm 2020-07-31-16-45 uploaded Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200724041508.QlTbrHnfh%akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=catalin.marinas@arm.com \
    --cc=hannes@cmpxchg.org \
    --cc=hdanton@sina.com \
    --cc=hughd@google.com \
    --cc=josef@toxicpanda.com \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mm-commits@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=will.deacon@arm.com \
    --cc=willy@infradead.org \
    --cc=xuyu@linux.alibaba.com \
    --cc=yang.shi@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

mm-commits Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/mm-commits/0 mm-commits/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 mm-commits mm-commits/ https://lore.kernel.org/mm-commits \
		mm-commits@vger.kernel.org
	public-inbox-index mm-commits

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.mm-commits


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git