From: Bibo Mao <maobibo@loongson.cn>
To: "Thomas Bogendoerfer" <tsbogend@alpha.franken.de>,
"Jiaxun Yang" <jiaxun.yang@flygoat.com>,
"Huacai Chen" <chenhc@lemote.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Paul Burton" <paulburton@kernel.org>,
"Dmitry Korotin" <dkorotin@wavecomp.com>,
"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
"Stafford Horne" <shorne@gmail.com>,
"Steven Price" <steven.price@arm.com>,
"Anshuman Khandual" <anshuman.khandual@arm.com>
Cc: linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org,
Mike Rapoport <rppt@linux.ibm.com>,
Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>,
"Maciej W. Rozycki" <macro@wdc.com>,
linux-mm@kvack.org, David Hildenbrand <david@redhat.com>
Subject: [PATCH v7 3/4] mm/memory.c: Add memory read privilege on page fault handling
Date: Wed, 27 May 2020 10:25:19 +0800 [thread overview]
Message-ID: <1590546320-21814-4-git-send-email-maobibo@loongson.cn> (raw)
In-Reply-To: <1590546320-21814-1-git-send-email-maobibo@loongson.cn>
Here add pte_sw_mkyoung function to make page readable on MIPS
platform during page fault handling. This patch improves page
fault latency about 10% on my MIPS machine with lmbench
lat_pagefault case.
It is noop function on other arches, there is no negative
influence on those architectures.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Acked-by: Andrew Morton <akpm@linux-foundation.org>
---
arch/mips/include/asm/pgtable.h | 2 ++
include/asm-generic/pgtable.h | 16 ++++++++++++++++
mm/memory.c | 3 +++
3 files changed, 21 insertions(+)
diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
index 6f40612..d9772aff 100644
--- a/arch/mips/include/asm/pgtable.h
+++ b/arch/mips/include/asm/pgtable.h
@@ -414,6 +414,8 @@ static inline pte_t pte_mkyoung(pte_t pte)
return pte;
}
+#define pte_sw_mkyoung pte_mkyoung
+
#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
static inline int pte_huge(pte_t pte) { return pte_val(pte) & _PAGE_HUGE; }
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index fa5c73f..b5278ec 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -244,6 +244,22 @@ static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addres
}
#endif
+/*
+ * On some architectures hardware does not set page access bit when accessing
+ * memory page, it is responsibilty of software setting this bit. It brings
+ * out extra page fault penalty to track page access bit. For optimization page
+ * access bit can be set during all page fault flow on these arches.
+ * To be differentiate with macro pte_mkyoung, this macro is used on platforms
+ * where software maintains page access bit.
+ */
+#ifndef pte_sw_mkyoung
+static inline pte_t pte_sw_mkyoung(pte_t pte)
+{
+ return pte;
+}
+#define pte_sw_mkyoung pte_sw_mkyoung
+#endif
+
#ifndef pte_savedwrite
#define pte_savedwrite pte_write
#endif
diff --git a/mm/memory.c b/mm/memory.c
index 8bb31c4..c7c8960 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2704,6 +2704,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
}
flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte));
entry = mk_pte(new_page, vma->vm_page_prot);
+ entry = pte_sw_mkyoung(entry);
entry = maybe_mkwrite(pte_mkdirty(entry), vma);
/*
* Clear the pte entry and flush it first, before updating the
@@ -3378,6 +3379,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
__SetPageUptodate(page);
entry = mk_pte(page, vma->vm_page_prot);
+ entry = pte_sw_mkyoung(entry);
if (vma->vm_flags & VM_WRITE)
entry = pte_mkwrite(pte_mkdirty(entry));
@@ -3660,6 +3662,7 @@ vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct mem_cgroup *memcg,
flush_icache_page(vma, page);
entry = mk_pte(page, vma->vm_page_prot);
+ entry = pte_sw_mkyoung(entry);
if (write)
entry = maybe_mkwrite(pte_mkdirty(entry), vma);
/* copy-on-write page */
--
1.8.3.1
next prev parent reply other threads:[~2020-05-27 2:26 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-27 2:25 [PATCH v7 0/4] MIPS: page fault handling optimization Bibo Mao
2020-05-27 2:25 ` [PATCH v7 1/4] MIPS: Do not flush tlb page when updating PTE entry Bibo Mao
2020-05-27 2:25 ` [PATCH v7 2/4] mm/memory.c: Update local TLB if PTE entry exists Bibo Mao
2020-05-28 19:23 ` Andrew Morton
2020-05-29 0:59 ` maobibo
2020-05-27 2:25 ` Bibo Mao [this message]
2020-05-27 2:25 ` [PATCH v7 4/4] MIPS: mm: add page valid judgement in function pte_modify Bibo Mao
2020-05-27 11:32 ` [PATCH v7 0/4] MIPS: page fault handling optimization Thomas Bogendoerfer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1590546320-21814-4-git-send-email-maobibo@loongson.cn \
--to=maobibo@loongson.cn \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=chenhc@lemote.com \
--cc=david@redhat.com \
--cc=dkorotin@wavecomp.com \
--cc=f4bug@amsat.org \
--cc=jiaxun.yang@flygoat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=macro@wdc.com \
--cc=paulburton@kernel.org \
--cc=rppt@linux.ibm.com \
--cc=sergei.shtylyov@cogentembedded.com \
--cc=shorne@gmail.com \
--cc=steven.price@arm.com \
--cc=tsbogend@alpha.franken.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).