From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754799AbbFKOJF (ORCPT ); Thu, 11 Jun 2015 10:09:05 -0400 Received: from mail-wi0-f172.google.com ([209.85.212.172]:37819 "EHLO mail-wi0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752875AbbFKOHv (ORCPT ); Thu, 11 Jun 2015 10:07:51 -0400 From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: linux-mml@vger.kernel.org, Andy Lutomirski , Andrew Morton , Denys Vlasenko , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H. Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , Waiman Long Subject: [PATCH 10/12] x86/mm: Make pgd_alloc()/pgd_free() lockless Date: Thu, 11 Jun 2015 16:07:15 +0200 Message-Id: <1434031637-9091-11-git-send-email-mingo@kernel.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1434031637-9091-1-git-send-email-mingo@kernel.org> References: <1434031637-9091-1-git-send-email-mingo@kernel.org> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The fork()/exit() code uses pgd_alloc()/pgd_free() to allocate/deallocate the PGD, with platform specific code setting up kernel pagetables. The x86 code uses a global pgd_list with an associated lock to update all PGDs of all tasks in the system synchronously. The lock is still kept to synchronize updates to all PGDs in the system, but all users of the list have been migrated to use the task list. So we can remove the pgd_list addition/removal from this code. The new PGD is private while constructed, so it needs no extra locking. Cc: Andrew Morton Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Cc: Waiman Long Cc: linux-mm@vger.kernel.org Signed-off-by: Ingo Molnar --- arch/x86/mm/pgtable.c | 27 +++------------------------ 1 file changed, 3 insertions(+), 24 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index e0bf90470d70..d855010a111f 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -125,22 +125,6 @@ static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd) swapper_pg_dir + KERNEL_PGD_BOUNDARY, KERNEL_PGD_PTRS); } - - /* list required to sync kernel mapping updates */ - if (!SHARED_KERNEL_PMD) { - pgd_set_mm(pgd, mm); - pgd_list_add(pgd); - } -} - -static void pgd_dtor(pgd_t *pgd) -{ - if (SHARED_KERNEL_PMD) - return; - - spin_lock(&pgd_lock); - pgd_list_del(pgd); - spin_unlock(&pgd_lock); } /* @@ -370,17 +354,13 @@ pgd_t *pgd_alloc(struct mm_struct *mm) goto out_free_pmds; /* - * Make sure that pre-populating the pmds is atomic with - * respect to anything walking the pgd_list, so that they - * never see a partially populated pgd. + * No locking is needed here, as the PGD is still private, + * so no code walking the task list and looking at mm->pgd + * will be able to see it before it's fully constructed: */ - spin_lock(&pgd_lock); - pgd_ctor(mm, pgd); pgd_prepopulate_pmd(mm, pgd, pmds); - spin_unlock(&pgd_lock); - return pgd; out_free_pmds: @@ -451,7 +431,6 @@ void arch_pgd_init_late(struct mm_struct *mm, pgd_t *pgd) void pgd_free(struct mm_struct *mm, pgd_t *pgd) { pgd_mop_up_pmds(mm, pgd); - pgd_dtor(pgd); paravirt_pgd_free(mm, pgd); _pgd_free(pgd); } -- 2.1.4