From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756789AbYAPWXA (ORCPT ); Wed, 16 Jan 2008 17:23:00 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754193AbYAPWPf (ORCPT ); Wed, 16 Jan 2008 17:15:35 -0500 Received: from ns.suse.de ([195.135.220.2]:55230 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753314AbYAPWPX (ORCPT ); Wed, 16 Jan 2008 17:15:23 -0500 From: Andi Kleen References: <200801161114.239449000@suse.de> In-Reply-To: <200801161114.239449000@suse.de> To: linux-kernel@vger.kernel.org, mingo@elte.hu, tglx@linutronix.de, jbeulich@novell.com, venkatesh.pallipadi@intel.com Subject: [PATCH] [22/36] CPA: Reorder TLB / cache flushes to follow Intel recommendation Message-Id: <20080116221521.BA00F150C8@wotan.suse.de> Date: Wed, 16 Jan 2008 23:15:21 +0100 (CET) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Intel recommends to first flush the TLBs and then the caches on caching attribute changes. c_p_a() previously did it the other way round. Reorder that. The procedure is still not fully compliant to the Intel documentation because Intel recommends a all CPU synchronization step between the TLB flushes and the cache flushes. However on all new Intel CPUs this is now meaningless anyways because they support Self-Snoop and can skip the cache flush step anyways. Signed-off-by: Andi Kleen Acked-by: Jan Beulich --- arch/x86/mm/pageattr_32.c | 13 ++++++++++--- arch/x86/mm/pageattr_64.c | 14 ++++++++++---- 2 files changed, 20 insertions(+), 7 deletions(-) Index: linux/arch/x86/mm/pageattr_32.c =================================================================== --- linux.orig/arch/x86/mm/pageattr_32.c +++ linux/arch/x86/mm/pageattr_32.c @@ -97,9 +97,6 @@ static void flush_kernel_map(void *arg) struct flush_arg *a = (struct flush_arg *)arg; struct flush *f; - if ((!cpu_has_clflush || a->full_flush) && boot_cpu_data.x86_model >= 4 && - !cpu_has_ss) - wbinvd(); list_for_each_entry(f, &a->l, l) { if (!a->full_flush && !cpu_has_ss) clflush_cache_range((void *)f->addr, PAGE_SIZE); @@ -112,6 +109,16 @@ static void flush_kernel_map(void *arg) (boot_cpu_data.x86_vendor == X86_VENDOR_AMD && boot_cpu_data.x86 == 7)) __flush_tlb_all(); + + /* + * RED-PEN: Intel documentation ask for a CPU synchronization step + * here and in the loop. But it is moot on Self-Snoop CPUs anyways. + */ + + if ((!cpu_has_clflush || a->full_flush) && + !cpu_has_ss && boot_cpu_data.x86_model >= 4) + wbinvd(); + } static void set_pmd_pte(pte_t *kpte, unsigned long address, pte_t pte) Index: linux/arch/x86/mm/pageattr_64.c =================================================================== --- linux.orig/arch/x86/mm/pageattr_64.c +++ linux/arch/x86/mm/pageattr_64.c @@ -94,16 +94,22 @@ static void flush_kernel_map(void *arg) /* When clflush is available always use it because it is much cheaper than WBINVD. */ - if ((a->full_flush || !cpu_has_clflush) && !cpu_has_ss) - wbinvd(); list_for_each_entry(f, &a->l, l) { - if (!a->full_flush && !cpu_has_ss) - clflush_cache_range((void *)f->addr, PAGE_SIZE); if (!a->full_flush) __flush_tlb_one(f->addr); + if (!a->full_flush && !cpu_has_ss) + clflush_cache_range((void *)f->addr, PAGE_SIZE); } if (a->full_flush) __flush_tlb_all(); + + /* + * RED-PEN: Intel documentation ask for a CPU synchronization step + * here and in the loop. But it is moot on Self-Snoop CPUs anyways. + */ + + if ((!cpu_has_clflush || a->full_flush) && !cpu_has_ss) + wbinvd(); } /* both protected by init_mm.mmap_sem */