From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752827AbdFOOx3 (ORCPT ); Thu, 15 Jun 2017 10:53:29 -0400 Received: from mga03.intel.com ([134.134.136.65]:10637 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752358AbdFOOx2 (ORCPT ); Thu, 15 Jun 2017 10:53:28 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.39,343,1493708400"; d="scan'208";a="99861836" From: "Kirill A. Shutemov" To: Andrew Morton , Vlastimil Babka , Vineet Gupta , Russell King , Will Deacon , Catalin Marinas , Ralf Baechle , "David S. Miller" , "Aneesh Kumar K . V" , Martin Schwidefsky , Heiko Carstens , Andrea Arcangeli Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , Ingo Molnar , "H . Peter Anvin" , Thomas Gleixner Subject: [PATCHv2 1/3] x86/mm: Provide pmdp_establish() helper Date: Thu, 15 Jun 2017 17:52:22 +0300 Message-Id: <20170615145224.66200-2-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170615145224.66200-1-kirill.shutemov@linux.intel.com> References: <20170615145224.66200-1-kirill.shutemov@linux.intel.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We need an atomic way to setup pmd page table entry, avoiding races with CPU setting dirty/accessed bits. This is required to implement pmdp_invalidate() that doesn't loose these bits. On PAE we have to use cmpxchg8b as we cannot assume what is value of new pmd and setting it up half-by-half can expose broken corrupted entry to CPU. Signed-off-by: Kirill A. Shutemov Cc: Ingo Molnar Cc: H. Peter Anvin Cc: Thomas Gleixner --- arch/x86/include/asm/pgtable-3level.h | 18 ++++++++++++++++++ arch/x86/include/asm/pgtable.h | 14 ++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/arch/x86/include/asm/pgtable-3level.h b/arch/x86/include/asm/pgtable-3level.h index 50d35e3185f5..471c8a851363 100644 --- a/arch/x86/include/asm/pgtable-3level.h +++ b/arch/x86/include/asm/pgtable-3level.h @@ -180,6 +180,24 @@ static inline pmd_t native_pmdp_get_and_clear(pmd_t *pmdp) #define native_pmdp_get_and_clear(xp) native_local_pmdp_get_and_clear(xp) #endif +#ifndef pmdp_establish +#define pmdp_establish pmdp_establish +static inline pmd_t pmdp_establish(pmd_t *pmdp, pmd_t pmd) +{ + pmd_t old; + + /* + * We cannot assume what is value of pmd here, so there's no easy way + * to set if half by half. We have to fall back to cmpxchg64. + */ + { + old = *pmdp; + } while (cmpxchg64(&pmdp->pmd, old.pmd, pmd.pmd) != old.pmd); + + return old; +} +#endif + #ifdef CONFIG_SMP union split_pud { struct { diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index f5af95a0c6b8..a924fc6a96b9 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1092,6 +1092,20 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, clear_bit(_PAGE_BIT_RW, (unsigned long *)pmdp); } +#ifndef pmdp_establish +#define pmdp_establish pmdp_establish +static inline pmd_t pmdp_establish(pmd_t *pmdp, pmd_t pmd) +{ + if (IS_ENABLED(CONFIG_SMP)) { + return xchg(pmdp, pmd); + } else { + pmd_t old = *pmdp; + *pmdp = pmd; + return old; + } +} +#endif + /* * clone_pgd_range(pgd_t *dst, pgd_t *src, int count); * -- 2.11.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Kirill A. Shutemov" Subject: [PATCHv2 1/3] x86/mm: Provide pmdp_establish() helper Date: Thu, 15 Jun 2017 17:52:22 +0300 Message-ID: <20170615145224.66200-2-kirill.shutemov@linux.intel.com> References: <20170615145224.66200-1-kirill.shutemov@linux.intel.com> Return-path: In-Reply-To: <20170615145224.66200-1-kirill.shutemov@linux.intel.com> Sender: owner-linux-mm@kvack.org To: Andrew Morton , Vlastimil Babka , Vineet Gupta , Russell King , Will Deacon , Catalin Marinas , Ralf Baechle , "David S. Miller" , "Aneesh Kumar K . V" , Martin Schwidefsky , Heiko Carstens , Andrea Arcangeli Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , Ingo Molnar , "H . Peter Anvin" , Thomas Gleixner List-Id: linux-arch.vger.kernel.org We need an atomic way to setup pmd page table entry, avoiding races with CPU setting dirty/accessed bits. This is required to implement pmdp_invalidate() that doesn't loose these bits. On PAE we have to use cmpxchg8b as we cannot assume what is value of new pmd and setting it up half-by-half can expose broken corrupted entry to CPU. Signed-off-by: Kirill A. Shutemov Cc: Ingo Molnar Cc: H. Peter Anvin Cc: Thomas Gleixner --- arch/x86/include/asm/pgtable-3level.h | 18 ++++++++++++++++++ arch/x86/include/asm/pgtable.h | 14 ++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/arch/x86/include/asm/pgtable-3level.h b/arch/x86/include/asm/pgtable-3level.h index 50d35e3185f5..471c8a851363 100644 --- a/arch/x86/include/asm/pgtable-3level.h +++ b/arch/x86/include/asm/pgtable-3level.h @@ -180,6 +180,24 @@ static inline pmd_t native_pmdp_get_and_clear(pmd_t *pmdp) #define native_pmdp_get_and_clear(xp) native_local_pmdp_get_and_clear(xp) #endif +#ifndef pmdp_establish +#define pmdp_establish pmdp_establish +static inline pmd_t pmdp_establish(pmd_t *pmdp, pmd_t pmd) +{ + pmd_t old; + + /* + * We cannot assume what is value of pmd here, so there's no easy way + * to set if half by half. We have to fall back to cmpxchg64. + */ + { + old = *pmdp; + } while (cmpxchg64(&pmdp->pmd, old.pmd, pmd.pmd) != old.pmd); + + return old; +} +#endif + #ifdef CONFIG_SMP union split_pud { struct { diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index f5af95a0c6b8..a924fc6a96b9 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1092,6 +1092,20 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, clear_bit(_PAGE_BIT_RW, (unsigned long *)pmdp); } +#ifndef pmdp_establish +#define pmdp_establish pmdp_establish +static inline pmd_t pmdp_establish(pmd_t *pmdp, pmd_t pmd) +{ + if (IS_ENABLED(CONFIG_SMP)) { + return xchg(pmdp, pmd); + } else { + pmd_t old = *pmdp; + *pmdp = pmd; + return old; + } +} +#endif + /* * clone_pgd_range(pgd_t *dst, pgd_t *src, int count); * -- 2.11.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org