From: "Justin He (Arm Technology China)" <Justin.He@arm.com> To: Marc Zyngier <maz@kernel.org>, Will Deacon <will@kernel.org> Cc: Catalin Marinas <Catalin.Marinas@arm.com>, Mark Rutland <Mark.Rutland@arm.com>, James Morse <James.Morse@arm.com>, Matthew Wilcox <willy@infradead.org>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, Punit Agrawal <punitagrawal@gmail.com>, Thomas Gleixner <tglx@linutronix.de>, Andrew Morton <akpm@linux-foundation.org>, "hejianet@gmail.com" <hejianet@gmail.com>, "Kaly Xin (Arm Technology China)" <Kaly.Xin@arm.com>, nd <nd@arm.com> Subject: RE: [PATCH v10 2/3] arm64: mm: implement arch_faults_on_old_pte() on arm64 Date: Tue, 8 Oct 2019 01:55:04 +0000 [thread overview] Message-ID: <DB7PR08MB308265EB3ED2465D2471B492F79A0@DB7PR08MB3082.eurprd08.prod.outlook.com> (raw) In-Reply-To: <20191001143219.018281be@why> Hi Will and Marc > -----Original Message----- > From: Marc Zyngier <maz@kernel.org> > Sent: 2019年10月1日 21:32 > To: Will Deacon <will@kernel.org> > Cc: Justin He (Arm Technology China) <Justin.He@arm.com>; Catalin > Marinas <Catalin.Marinas@arm.com>; Mark Rutland > <Mark.Rutland@arm.com>; James Morse <James.Morse@arm.com>; > Matthew Wilcox <willy@infradead.org>; Kirill A. Shutemov > <kirill.shutemov@linux.intel.com>; linux-arm-kernel@lists.infradead.org; > linux-kernel@vger.kernel.org; linux-mm@kvack.org; Punit Agrawal > <punitagrawal@gmail.com>; Thomas Gleixner <tglx@linutronix.de>; > Andrew Morton <akpm@linux-foundation.org>; hejianet@gmail.com; Kaly > Xin (Arm Technology China) <Kaly.Xin@arm.com> > Subject: Re: [PATCH v10 2/3] arm64: mm: implement > arch_faults_on_old_pte() on arm64 > > On Tue, 1 Oct 2019 13:50:32 +0100 > Will Deacon <will@kernel.org> wrote: > > > On Mon, Sep 30, 2019 at 09:57:39AM +0800, Jia He wrote: > > > On arm64 without hardware Access Flag, copying fromuser will fail > because > > > the pte is old and cannot be marked young. So we always end up with > zeroed > > > page after fork() + CoW for pfn mappings. we don't always have a > > > hardware-managed access flag on arm64. > > > > > > Hence implement arch_faults_on_old_pte on arm64 to indicate that it > might > > > cause page fault when accessing old pte. > > > > > > Signed-off-by: Jia He <justin.he@arm.com> > > > Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> > > > --- > > > arch/arm64/include/asm/pgtable.h | 14 ++++++++++++++ > > > 1 file changed, 14 insertions(+) > > > > > > diff --git a/arch/arm64/include/asm/pgtable.h > b/arch/arm64/include/asm/pgtable.h > > > index 7576df00eb50..e96fb82f62de 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -885,6 +885,20 @@ static inline void update_mmu_cache(struct > vm_area_struct *vma, > > > #define phys_to_ttbr(addr) (addr) > > > #endif > > > > > > +/* > > > + * On arm64 without hardware Access Flag, copying from user will fail > because > > > + * the pte is old and cannot be marked young. So we always end up > with zeroed > > > + * page after fork() + CoW for pfn mappings. We don't always have a > > > + * hardware-managed access flag on arm64. > > > + */ > > > +static inline bool arch_faults_on_old_pte(void) > > > +{ > > > + WARN_ON(preemptible()); > > > + > > > + return !cpu_has_hw_af(); > > > +} > > > > Does this work correctly in a KVM guest? (i.e. is the MMFR sanitised in > that > > case, despite not being the case on the host?) > > Yup, all the 64bit MMFRs are trapped (HCR_EL2.TID3 is set for an > AArch64 guest), and we return the sanitised version. Thanks for Marc's explanation. I verified the patch series on a kvm guest (-M virt) with simulated nvdimm device created by qemu. The host is ThunderX2 aarch64. > > But that's an interesting remark: we're now trading an extra fault on > CPUs that do not support HWAFDBS for a guaranteed trap for each and > every guest under the sun that will hit the COW path... > > My gut feeling is that this is going to be pretty visible. Jia, do you > have any numbers for this kind of behaviour? It is not a common COW path, but a COW for PFN mapping pages only. I add a g_counter before pte_mkyoung in force_mkyoung{} when testing vmmalloc_fork at [1]. In this test case, it will start M fork processes and N pthreads. The default is M=2,N=4. the g_counter is about 241, that is it will hit my patch series for 241 times. If I set M=20 and N=40 for TEST3, the g_counter is about 1492. [1] https://github.com/pmem/pmdk/tree/master/src/test/vmmalloc_fork -- Cheers, Justin (Jia He)
WARNING: multiple messages have this Message-ID (diff)
From: "Justin He (Arm Technology China)" <Justin.He@arm.com> To: Marc Zyngier <maz@kernel.org>, Will Deacon <will@kernel.org> Cc: Mark Rutland <Mark.Rutland@arm.com>, "Kaly Xin \(Arm Technology China\)" <Kaly.Xin@arm.com>, Catalin Marinas <Catalin.Marinas@arm.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Matthew Wilcox <willy@infradead.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, James Morse <James.Morse@arm.com>, "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>, Punit Agrawal <punitagrawal@gmail.com>, "hejianet@gmail.com" <hejianet@gmail.com>, Thomas Gleixner <tglx@linutronix.de>, nd <nd@arm.com>, Andrew Morton <akpm@linux-foundation.org>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Subject: RE: [PATCH v10 2/3] arm64: mm: implement arch_faults_on_old_pte() on arm64 Date: Tue, 8 Oct 2019 01:55:04 +0000 [thread overview] Message-ID: <DB7PR08MB308265EB3ED2465D2471B492F79A0@DB7PR08MB3082.eurprd08.prod.outlook.com> (raw) In-Reply-To: <20191001143219.018281be@why> Hi Will and Marc > -----Original Message----- > From: Marc Zyngier <maz@kernel.org> > Sent: 2019年10月1日 21:32 > To: Will Deacon <will@kernel.org> > Cc: Justin He (Arm Technology China) <Justin.He@arm.com>; Catalin > Marinas <Catalin.Marinas@arm.com>; Mark Rutland > <Mark.Rutland@arm.com>; James Morse <James.Morse@arm.com>; > Matthew Wilcox <willy@infradead.org>; Kirill A. Shutemov > <kirill.shutemov@linux.intel.com>; linux-arm-kernel@lists.infradead.org; > linux-kernel@vger.kernel.org; linux-mm@kvack.org; Punit Agrawal > <punitagrawal@gmail.com>; Thomas Gleixner <tglx@linutronix.de>; > Andrew Morton <akpm@linux-foundation.org>; hejianet@gmail.com; Kaly > Xin (Arm Technology China) <Kaly.Xin@arm.com> > Subject: Re: [PATCH v10 2/3] arm64: mm: implement > arch_faults_on_old_pte() on arm64 > > On Tue, 1 Oct 2019 13:50:32 +0100 > Will Deacon <will@kernel.org> wrote: > > > On Mon, Sep 30, 2019 at 09:57:39AM +0800, Jia He wrote: > > > On arm64 without hardware Access Flag, copying fromuser will fail > because > > > the pte is old and cannot be marked young. So we always end up with > zeroed > > > page after fork() + CoW for pfn mappings. we don't always have a > > > hardware-managed access flag on arm64. > > > > > > Hence implement arch_faults_on_old_pte on arm64 to indicate that it > might > > > cause page fault when accessing old pte. > > > > > > Signed-off-by: Jia He <justin.he@arm.com> > > > Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> > > > --- > > > arch/arm64/include/asm/pgtable.h | 14 ++++++++++++++ > > > 1 file changed, 14 insertions(+) > > > > > > diff --git a/arch/arm64/include/asm/pgtable.h > b/arch/arm64/include/asm/pgtable.h > > > index 7576df00eb50..e96fb82f62de 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -885,6 +885,20 @@ static inline void update_mmu_cache(struct > vm_area_struct *vma, > > > #define phys_to_ttbr(addr) (addr) > > > #endif > > > > > > +/* > > > + * On arm64 without hardware Access Flag, copying from user will fail > because > > > + * the pte is old and cannot be marked young. So we always end up > with zeroed > > > + * page after fork() + CoW for pfn mappings. We don't always have a > > > + * hardware-managed access flag on arm64. > > > + */ > > > +static inline bool arch_faults_on_old_pte(void) > > > +{ > > > + WARN_ON(preemptible()); > > > + > > > + return !cpu_has_hw_af(); > > > +} > > > > Does this work correctly in a KVM guest? (i.e. is the MMFR sanitised in > that > > case, despite not being the case on the host?) > > Yup, all the 64bit MMFRs are trapped (HCR_EL2.TID3 is set for an > AArch64 guest), and we return the sanitised version. Thanks for Marc's explanation. I verified the patch series on a kvm guest (-M virt) with simulated nvdimm device created by qemu. The host is ThunderX2 aarch64. > > But that's an interesting remark: we're now trading an extra fault on > CPUs that do not support HWAFDBS for a guaranteed trap for each and > every guest under the sun that will hit the COW path... > > My gut feeling is that this is going to be pretty visible. Jia, do you > have any numbers for this kind of behaviour? It is not a common COW path, but a COW for PFN mapping pages only. I add a g_counter before pte_mkyoung in force_mkyoung{} when testing vmmalloc_fork at [1]. In this test case, it will start M fork processes and N pthreads. The default is M=2,N=4. the g_counter is about 241, that is it will hit my patch series for 241 times. If I set M=20 and N=40 for TEST3, the g_counter is about 1492. [1] https://github.com/pmem/pmdk/tree/master/src/test/vmmalloc_fork -- Cheers, Justin (Jia He) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-10-08 1:55 UTC|newest] Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-09-30 1:57 [PATCH v10 0/3] fix double page fault on arm64 Jia He 2019-09-30 1:57 ` Jia He 2019-09-30 1:57 ` [PATCH v10 1/3] arm64: cpufeature: introduce helper cpu_has_hw_af() Jia He 2019-09-30 1:57 ` Jia He 2019-10-01 12:54 ` Will Deacon 2019-10-01 12:54 ` Will Deacon 2019-10-01 13:18 ` Marc Zyngier 2019-10-01 13:18 ` Marc Zyngier 2019-10-08 1:12 ` Justin He (Arm Technology China) 2019-10-08 1:12 ` Justin He (Arm Technology China) 2019-10-08 1:12 ` Justin He (Arm Technology China) 2019-10-08 15:32 ` Suzuki K Poulose 2019-10-08 15:32 ` Suzuki K Poulose 2019-10-08 15:32 ` Suzuki K Poulose 2019-10-09 6:29 ` Jia He 2019-10-09 6:29 ` Jia He 2019-10-09 6:29 ` Jia He 2019-09-30 1:57 ` [PATCH v10 2/3] arm64: mm: implement arch_faults_on_old_pte() on arm64 Jia He 2019-09-30 1:57 ` Jia He 2019-10-01 12:50 ` Will Deacon 2019-10-01 12:50 ` Will Deacon 2019-10-01 13:32 ` Marc Zyngier 2019-10-01 13:32 ` Marc Zyngier 2019-10-08 1:55 ` Justin He (Arm Technology China) [this message] 2019-10-08 1:55 ` Justin He (Arm Technology China) 2019-10-08 1:55 ` Justin He (Arm Technology China) 2019-10-08 2:30 ` Justin He (Arm Technology China) 2019-10-08 2:30 ` Justin He (Arm Technology China) 2019-10-08 2:30 ` Justin He (Arm Technology China) 2019-10-08 7:46 ` Marc Zyngier 2019-10-08 7:46 ` Marc Zyngier 2019-10-08 7:46 ` Marc Zyngier 2019-09-30 1:57 ` [PATCH v10 3/3] mm: fix double page fault on arm64 if PTE_AF is cleared Jia He 2019-09-30 1:57 ` Jia He 2019-10-01 12:54 ` Will Deacon 2019-10-01 12:54 ` Will Deacon 2019-10-08 2:19 ` Justin He (Arm Technology China) 2019-10-08 2:19 ` Justin He (Arm Technology China) 2019-10-08 2:19 ` Justin He (Arm Technology China) 2019-10-08 12:39 ` Will Deacon 2019-10-08 12:39 ` Will Deacon 2019-10-08 12:39 ` Will Deacon 2019-10-08 12:58 ` Justin He (Arm Technology China) 2019-10-08 12:58 ` Justin He (Arm Technology China) 2019-10-08 12:58 ` Justin He (Arm Technology China) 2019-10-08 14:32 ` Kirill A. Shutemov 2019-10-08 14:32 ` Kirill A. Shutemov 2019-10-08 14:32 ` Kirill A. Shutemov 2019-10-16 23:21 ` Palmer Dabbelt 2019-10-16 23:21 ` Palmer Dabbelt 2019-10-16 23:46 ` Will Deacon 2019-10-16 23:46 ` Will Deacon 2019-10-18 20:38 ` Palmer Dabbelt 2019-10-18 20:38 ` Palmer Dabbelt 2019-10-19 2:59 ` Jia He 2019-10-19 2:59 ` Jia He
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=DB7PR08MB308265EB3ED2465D2471B492F79A0@DB7PR08MB3082.eurprd08.prod.outlook.com \ --to=justin.he@arm.com \ --cc=Catalin.Marinas@arm.com \ --cc=James.Morse@arm.com \ --cc=Kaly.Xin@arm.com \ --cc=Mark.Rutland@arm.com \ --cc=akpm@linux-foundation.org \ --cc=hejianet@gmail.com \ --cc=kirill.shutemov@linux.intel.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=maz@kernel.org \ --cc=nd@arm.com \ --cc=punitagrawal@gmail.com \ --cc=tglx@linutronix.de \ --cc=will@kernel.org \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.