From: Ingo Molnar <mingo@kernel.org> To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andy Lutomirski <luto@amacapital.net>, Andrew Morton <akpm@linux-foundation.org>, Denys Vlasenko <dvlasenk@redhat.com>, Brian Gerst <brgerst@gmail.com>, Peter Zijlstra <peterz@infradead.org>, Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>, Linus Torvalds <torvalds@linux-foundation.org>, Oleg Nesterov <oleg@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Waiman Long <Waiman.Long@hp.com> Subject: [PATCH 12/12] x86/mm: Simplify pgd_alloc() Date: Sat, 13 Jun 2015 11:49:15 +0200 [thread overview] Message-ID: <1434188955-31397-13-git-send-email-mingo@kernel.org> (raw) In-Reply-To: <1434188955-31397-1-git-send-email-mingo@kernel.org> Right now pgd_alloc() uses pgd_ctor(), which copies over the current swapper_pg_dir[] to a new task's PGD. This is not necessary, it's enough if we clear it: the PGD will then be properly updated by arch_pgd_init_late(). Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Waiman Long <Waiman.Long@hp.com> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/mm/pgtable.c | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 7cca42c01677..d7d341e57e33 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -87,20 +87,6 @@ void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud) #define UNSHARED_PTRS_PER_PGD \ (SHARED_KERNEL_PMD ? KERNEL_PGD_BOUNDARY : PTRS_PER_PGD) -static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd) -{ - /* If the pgd points to a shared pagetable level (either the - ptes in non-PAE, or shared PMD in PAE), then just copy the - references from swapper_pg_dir. */ - if (CONFIG_PGTABLE_LEVELS == 2 || - (CONFIG_PGTABLE_LEVELS == 3 && SHARED_KERNEL_PMD) || - CONFIG_PGTABLE_LEVELS == 4) { - clone_pgd_range(pgd + KERNEL_PGD_BOUNDARY, - swapper_pg_dir + KERNEL_PGD_BOUNDARY, - KERNEL_PGD_PTRS); - } -} - /* * List of all pgd's needed for non-PAE so it can invalidate entries * in both cached and uncached pgd's; not needed for PAE since the @@ -328,11 +314,16 @@ pgd_t *pgd_alloc(struct mm_struct *mm) goto out_free_pmds; /* - * No locking is needed here, as the PGD is still private, - * so no code walking the task list and looking at mm->pgd - * will be able to see it before it's fully constructed: + * Zero out the kernel portion here, we'll set them up in + * arch_pgd_init_late(), when the pgd is globally + * visible already per the task list, so that it cannot + * miss updates. + * + * We need to zero it here, to make sure arch_pgd_init_late() + * can initialize them without locking. */ - pgd_ctor(mm, pgd); + memset(pgd + KERNEL_PGD_BOUNDARY, 0, KERNEL_PGD_PTRS*sizeof(pgd_t)); + pgd_prepopulate_pmd(mm, pgd, pmds); return pgd; -- 2.1.4
WARNING: multiple messages have this Message-ID (diff)
From: Ingo Molnar <mingo@kernel.org> To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andy Lutomirski <luto@amacapital.net>, Andrew Morton <akpm@linux-foundation.org>, Denys Vlasenko <dvlasenk@redhat.com>, Brian Gerst <brgerst@gmail.com>, Peter Zijlstra <peterz@infradead.org>, Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>, Linus Torvalds <torvalds@linux-foundation.org>, Oleg Nesterov <oleg@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Waiman Long <Waiman.Long@hp.com> Subject: [PATCH 12/12] x86/mm: Simplify pgd_alloc() Date: Sat, 13 Jun 2015 11:49:15 +0200 [thread overview] Message-ID: <1434188955-31397-13-git-send-email-mingo@kernel.org> (raw) In-Reply-To: <1434188955-31397-1-git-send-email-mingo@kernel.org> Right now pgd_alloc() uses pgd_ctor(), which copies over the current swapper_pg_dir[] to a new task's PGD. This is not necessary, it's enough if we clear it: the PGD will then be properly updated by arch_pgd_init_late(). Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Waiman Long <Waiman.Long@hp.com> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/mm/pgtable.c | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 7cca42c01677..d7d341e57e33 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -87,20 +87,6 @@ void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud) #define UNSHARED_PTRS_PER_PGD \ (SHARED_KERNEL_PMD ? KERNEL_PGD_BOUNDARY : PTRS_PER_PGD) -static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd) -{ - /* If the pgd points to a shared pagetable level (either the - ptes in non-PAE, or shared PMD in PAE), then just copy the - references from swapper_pg_dir. */ - if (CONFIG_PGTABLE_LEVELS == 2 || - (CONFIG_PGTABLE_LEVELS == 3 && SHARED_KERNEL_PMD) || - CONFIG_PGTABLE_LEVELS == 4) { - clone_pgd_range(pgd + KERNEL_PGD_BOUNDARY, - swapper_pg_dir + KERNEL_PGD_BOUNDARY, - KERNEL_PGD_PTRS); - } -} - /* * List of all pgd's needed for non-PAE so it can invalidate entries * in both cached and uncached pgd's; not needed for PAE since the @@ -328,11 +314,16 @@ pgd_t *pgd_alloc(struct mm_struct *mm) goto out_free_pmds; /* - * No locking is needed here, as the PGD is still private, - * so no code walking the task list and looking at mm->pgd - * will be able to see it before it's fully constructed: + * Zero out the kernel portion here, we'll set them up in + * arch_pgd_init_late(), when the pgd is globally + * visible already per the task list, so that it cannot + * miss updates. + * + * We need to zero it here, to make sure arch_pgd_init_late() + * can initialize them without locking. */ - pgd_ctor(mm, pgd); + memset(pgd + KERNEL_PGD_BOUNDARY, 0, KERNEL_PGD_PTRS*sizeof(pgd_t)); + pgd_prepopulate_pmd(mm, pgd, pmds); return pgd; -- 2.1.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-06-13 9:51 UTC|newest] Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-06-13 9:49 [PATCH 00/12, v2] x86/mm: Implement lockless pgd_alloc()/pgd_free() Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 01/12] x86/mm/pat: Don't free PGD entries on memory unmap Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 02/12] x86/mm/hotplug: Remove pgd_list use from the memory hotplug code Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 19:24 ` Oleg Nesterov 2015-06-13 19:24 ` Oleg Nesterov 2015-06-14 7:36 ` Ingo Molnar 2015-06-14 7:36 ` Ingo Molnar 2015-06-14 19:24 ` Oleg Nesterov 2015-06-14 19:24 ` Oleg Nesterov 2015-06-14 19:38 ` Oleg Nesterov 2015-06-14 19:38 ` Oleg Nesterov 2015-06-15 0:40 ` Paul E. McKenney 2015-06-15 0:40 ` Paul E. McKenney 2015-06-15 20:33 ` Ingo Molnar 2015-06-15 20:33 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 03/12] x86/mm/hotplug: Don't remove PGD entries in remove_pagetable() Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 04/12] x86/mm/hotplug: Simplify sync_global_pgds() Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 05/12] mm: Introduce arch_pgd_init_late() Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 06/12] x86/mm: Enable and use the arch_pgd_init_late() method Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 07/12] x86/virt/guest/xen: Remove use of pgd_list from the Xen guest code Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-14 8:26 ` Ingo Molnar 2015-06-14 8:26 ` Ingo Molnar 2015-06-15 9:05 ` Ian Campbell 2015-06-15 9:05 ` Ian Campbell 2015-06-15 9:05 ` Ian Campbell 2015-06-15 10:30 ` David Vrabel 2015-06-15 10:30 ` David Vrabel 2015-06-15 20:35 ` Ingo Molnar 2015-06-15 20:35 ` Ingo Molnar 2015-06-15 20:35 ` Ingo Molnar 2015-06-16 14:15 ` David Vrabel 2015-06-16 14:15 ` David Vrabel 2015-06-16 14:15 ` David Vrabel 2015-06-16 14:19 ` Boris Ostrovsky 2015-06-16 14:19 ` Boris Ostrovsky 2015-06-16 14:19 ` Boris Ostrovsky 2015-06-16 14:27 ` David Vrabel 2015-06-16 14:27 ` David Vrabel 2015-06-16 14:27 ` David Vrabel 2015-06-15 10:30 ` David Vrabel 2015-06-13 9:49 ` [PATCH 08/12] x86/mm: Remove pgd_list use from vmalloc_sync_all() Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 09/12] x86/mm/pat/32: Remove pgd_list use from the PAT code Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 10/12] x86/mm: Make pgd_alloc()/pgd_free() lockless Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` [PATCH 11/12] x86/mm: Remove pgd_list leftovers Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar 2015-06-13 9:49 ` Ingo Molnar [this message] 2015-06-13 9:49 ` [PATCH 12/12] x86/mm: Simplify pgd_alloc() Ingo Molnar 2015-06-13 18:58 ` why do we need vmalloc_sync_all? Oleg Nesterov 2015-06-13 18:58 ` Oleg Nesterov 2015-06-14 7:59 ` Ingo Molnar 2015-06-14 7:59 ` Ingo Molnar 2015-06-14 20:06 ` Oleg Nesterov 2015-06-14 20:06 ` Oleg Nesterov 2015-06-15 2:47 ` Andi Kleen 2015-06-15 2:47 ` Andi Kleen 2015-06-15 2:57 ` Andy Lutomirski 2015-06-15 2:57 ` Andy Lutomirski 2015-06-15 20:28 ` Ingo Molnar 2015-06-15 20:28 ` Ingo Molnar 2015-06-15 20:48 ` Andy Lutomirski 2015-06-15 20:48 ` Andy Lutomirski -- strict thread matches above, loose matches on Subject: below -- 2015-06-11 14:07 [RFC PATCH 00/12] x86/mm: Implement lockless pgd_alloc()/pgd_free() Ingo Molnar 2015-06-11 14:07 ` [PATCH 12/12] x86/mm: Simplify pgd_alloc() Ingo Molnar
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1434188955-31397-13-git-send-email-mingo@kernel.org \ --to=mingo@kernel.org \ --cc=Waiman.Long@hp.com \ --cc=akpm@linux-foundation.org \ --cc=bp@alien8.de \ --cc=brgerst@gmail.com \ --cc=dvlasenk@redhat.com \ --cc=hpa@zytor.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=luto@amacapital.net \ --cc=oleg@redhat.com \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=torvalds@linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.